People usually don't think about the times that the ads haven't matched up.
For example a person I know got an ad for a specific ice cream she was talking about earlier. What she didn't think about is that it was summer and ice cream ads are incredibly common in summer and so is talking about ice cream.
She likely got many similar ads the days before and after, but she didn't think about them because they didn't match her theory that phones (specifically Facebook in her case) are listening.
I personally think it's extremely unlikely that phones are listening anywhere close to that degree. At absolutely worst they might try and gauge your mood or something, but that feels unlikely too.
First of all there would be a lot of actual evidence (like with network sniffing) of it happening if it were happening and the public and legal fallout that would come after someone figured it out would be enormous