Drift is the shifting of our historical job counts as previously undetectable job URLs come back online. This is reflected in the consistent downward trend in our dataset’s overall job count for the most recent 10 days.

There are several reasons jobs can temporarily disappear from a career page. The site could go down temporarily for maintenance, they could be formatted incorrectly rendering them “unscrapable” for a time, our scrape system could have an issue with a certain CID or the company could have changed its ATS or career portal software.

On the day those jobs disappear or otherwise undetectable by our spiders, they are reported as deleted. However, because we collapse on job URL, when the exact same URL re-appears on the page at a later date, we consider that the same job, and the delete date is removed from the record, showing that the job was never taken down. This is good because we are not duplicating records from the same jobs, but it causes what we call drift.