Missing contacts using the contact_list API

Hi @Mike80222! I’m not aware of this as a trending issue, however, I did escalate this internally. Could you provide an example so our team can properly follow up?

Hi @ActiveCampaign, could you give me a little more of an idea what you would consider an “example.” As I mentioned to the support person I talked to, it would probably take a bit of work to replicate it on your end. Bear in mind that there’s some code involved with looping through the contacts list (which I tend to do in python). I don’t think that you probably want to read through all my code.

If there is really a problem with duplicate/missing contacts in the API, I would think that someone else would have seen it. That is, assuming that anyone but me is actually downloading a substantial number of contacts with V1.

I can say that even when I ask for the contacts to be ordered by ID, that some of them come out of order. That seems to happen every time. But the duplicate/missing ones only seem to happen “sometimes.”

I could do a little more work to try to isolate when that happens. It almost seems like it depends on the time of day.

This all started, maybe, something like a week or two ago (maybe three). Before that it’s been working with no serious problems since 2020. I don’t suppose anything has changed in the V1 interface recently, right?

It was suggested that I move to the V3 interface. I’ve been looking at that, but it doesn’t look like it would be very easy to get all the info that I’m getting from V1 in anything close to the same format.

Hey Mike,

Hope you had a great weekend! Just wanted to pop in with an update to make sure we don’t leave you hanging. I see you’ve already connected with our technical team, but it seems you’re not quite getting the data you need from the suggested API v3. Would you please let me know what data is missing so that we can review? Thank you, Mike, I appreciate your time and efforts and look forward to working with you further. Talk soon!

(P.S. Apologies for the double post, was accidentally logged in as my test user - oops!)

I would actually rather not have to switch APIs. But I agree that it probably makes sense to work on duplicating what I have with V3, since that’s the one you’re presumably going to be supporting in the long run.

The issue I have is that I have a fair amount of code that I’ve been using for the last few years which processes things in the format used by V1. I don’t think there’s anything you can do to help me with that. But one bit of information you could tell me: In V1, for lists and tags and such, if provides the name. V3 appears to give numeric codes.

In a recent question someone else asked I saw this call recommended:

/api/3/fields

I haven’t actually tried it yet. Will that give me the info I need to translate those numeric values into list and tag names, and such?

Back to V1 for a second. Do you believe that using sort=id with api_action=contact_list will really sort them correctly? I’ve got about 2,700 contacts and I typically get about 500 of them out of order. Does this surprise you?

And, most of all, does it surprise you that I appear to be seeing duplicated and missing contacts in V1. I never really got a clear answer on that. Just the advice to switch to V3.

Appreciate you taking the time to talk to me about this. I feel like support here has really improved lately.

Mike

Hey Mike,

Thanks for following up and providing this additional info. Glad to be connected!

/api/3/fields will list the title and field ID of each field, so that would be a great option to use to correlate field titles with IDs. More info on that call can be found here: List all custom fields.

Similarly, there are also the following v3 calls for tags and lists:

• /api/3/tags List all tags
• /api/3/lists Retrieve all lists

Regarding v1, it does not sound normal to me that the results would be out of order. I would consider that surprising, as one would expect the results to be in the same order as defined in the call.

I did some digging and was able to find some info that might be relevant here. API v1’s contact_list and API v3’s List all contacts calls will not consider contacts that are not in a list. Contacts will only be part of the response if they are in a list (regardless of status).

Might that explain some of the missing contacts you’re seeing with the v1 calls?

If not, would you have any example contacts that are missing/duplicated, and/or any screenshots of your results showing the discrepancy? I’d love to send that up to our engineers to take a closer look.

Thank you so much!

First, thanks for the info on the V3 calls. I’ll take a look at that.

I did some digging and was able to find some info that might be relevant here. API v1’s contact_list and API v3’s List all contacts calls will not consider contacts that are not in a list. Contacts will only be part of the response if they are in a list (regardless of status).

I saw that too, but it’s not really relevant. It just means that it won’t give you contacts that have never been on any list. I’m not sure how I would even get contacts like that and I don’t have any. When V1 works, it gives all of them.

I have the feeling that nobody there actually uses V1 right? You don’t have any want to run a test on your end to see if you get them all in order – is that right?

would you have any example contacts that are missing/duplicated …

As I’ve mentioned before, I’m not clear on exactly what I can send you. I could send a list of contact IDs that were duplicated or missing on one particular day. But what are you going to do with that?

I could send you my Python code, but do you really want to see it?

You understand that this in not easily duplicated in detail, right? The V1 contact-list worked 99% of the time since I first started using it in 2020. Then something like three weeks ago it stared failing about 30% of the time. It hasn’t failed in my nightly runs in the past couple of days.

I was hoping that V1 was still actively supported and that you guys had some sort of test rig on your end – so you could definitively say “it works for us” or maybe you would see something weird. But I do understand that you can’t continually be testing every version of the interface. So if you don’t have any other ideas about V1 that’s OK. I am planning to move over to V3 as I get time, and I can live with periodic failures of V1 in the meantime if necessary.

Hey Mike,

Thanks for the additional context. I want to make sure I am getting you the most accurate and helpful assistance without additional delays, so I have reached out to my technical team to take a look and provide their input + run any tests. Please stay tuned and I will drop in with another update as soon as I have one. Thank you so much!

Hey Mike,

Thanks for your patience. I have engaged with my technical team and they would very much like to look into this further for you to understand what might be going on. They will need some details, but fortunately I have specific items to collect this time. Please supply the following:

1. Full API request URL (including all parameters)
2. If it’s a POST request, the POST data
3. The expected response/action
4. The actual response received from the server, and action performed (if any)

I recognize you might not want to share everything in this public thread, so feel free to send it to me in a private message.

Thank you so much and I look forward to getting to the bottom of this. Talk soon!

https://MY_UID.api-us1.com/admin/api.php?api_action=contact_list&api_key=MY_KEY&api_output=json&ids=ALL&full=1&page=PAGE_NUMBER&sort=id&sort_direction=ASC

I loop through this, incrementing PAGE_NUMBER as I go. Normally I get back exactly what the documentation says except that the contact IDs are not always in order. That’s the only repeatable surprise. But then sometimes some of the contacts are duplicated and missing. Note that this never happened from 2020 until about three weeks ago. And it hasn’t happened in the last three days. But the out of order ids is repeatable.

On the V3 front. I’m making progress. For my use case at least, V3 is much more complicated than V1. I’ve managed to get the info I need about mailing lists and tags, but I’m having trouble with what were called “actions” in V1. For example, here’s a snippit of what I get (for each contact) when I download contacts with V1:

“actions”: [
{
“text”: “Contact clicked a link …”,
“type”: “click”,
“tstamp”: “2024-07-03T23:48:02-05:00”
},
{
“text”: “Contact opened campaign …”,
“type”: “open”,
“tstamp”: “2024-07-03T23:47:47-05:00”
},

I’m getting both “activities” and side-loading “contactLogs” but neither of these appears to have the same info. Can you tell me how to get the info above from V3?

Just so you know, I haven’t seen the V1 download fail for several days now, but I would still be interested to hear if you see V1 showing contacts out of order (which I see every time).

Thanks!

Thanks for the update, Mike. I am glad to hear that there haven’t been any failures recently. I have passed this info along to our technical team and am waiting to hear back with more info. I will amend that request and add this question about the “actions” vs “activies” and “contactLogs” so that we can get some clear answers there. Please stay tuned. Thank you so much!

Just FYI, there were some contacts missing this morning. So the duplicated/missing thing does still happen. Just not as frequently. My thought on the out of sequence contacts (which always happens) was that if that was fixed, it might just fix the other non-repeatable problem. Since if they’re delivered in sequence perhaps there wouldn’t be duplicates. And (from what I’ve seen) there are only contacts missing when there are duplicates. Just something to pass on to … whoever you’re passing these things on to.

Should I just give up on this thread?

Hi Mike,

Thanks for following up. I just checked in on the ticket, and it looks like engineering is still investigating the issue. My apologies for the delay, I’m hopeful we’ll have an update soon. I do see some notes that I can provide in the meantime while we wait to hear more:

The “actions” from the v1 call correspond to the “recent activities” of the contact, which are returned by the api/3/activities endpoint in v3: https://developers.activecampaign.com/reference/list-contact-activities. You can use the available query parameters (specified in the documentation) to narrow down the information returned.

I hope that helps in the meantime and please stay tuned once we have additional updates. Thank you!

OK, well that doesn’t address any of my questions, but I’ll await further information.

Thanks for your reply, Mike. Looks like that was intended to answer this question you posed in a previous comment:

I’m getting both “activities” and side-loading “contactLogs” but neither of these appears to have the same info. Can you tell me how to get the info above from V3?

Hmm… I think that line you’re quoting is missing some context, but it’s quite possible that I’m missing something. It’s been a while now since I looked at that. I’ll have to take another look. Thanks for following up on that!

OK, I’ve retraced my steps here. I haven’t even tried to use the list-contact-activities because it says in the documentation:

“Contact ID parameter is now required: A contact ID is required to access this endpoint. This endpoint does not support multiple contact IDs with a single call.”

So, unless I’m misunderstanding something here, that would mean I’d have to do a separate API call for every single contact. Just for that information. That’s thousands of API calls, in addition to everything else. But perhaps I’m reading that wrong?

Understood, thanks for the context. I do see that it only supports one contact ID. I just pinged my engineering team back to check in on this and see if that can be circumvented.

I also want to confirm that we are still investigating the original issue. It sounds like it may take some time, so please stay tuned, and I will follow up with you as soon as I have an update. I do apologize for the inconvenience in the meantime.

Thank you!