That's like the most trivial of theories one can test for.

Save response time every minute

while true; do /usr/bin/time -f "%e `date`" dig '@1.1.1.1' +noall programming.dev &>>/tmp/dns_clf_perf.txt; sleep 60; done

Then after a while (maybe a couple of days), check the worst numbers:

sort -n /tmp/dns_clf_perf.txt |tail

Run the same script with a different DNS server at the same time, and compare numbers.

Dates included in case there are any patterns regarding the time of day/week.