SEO Scanner Improvements & Multi-Axis Pagination — May 13, 2026

Today I worked on the SEO scanner, my main goals being to improve the pagination system and address some outstanding issues with our data. My goal was to increase our domain ceiling from about 394 to about 714. I’m happy to say that I successfully shipped a multi-axis paginator that will help us with that goal. While I only had a standard 8-hour work day, I did manage to address a couple of large roadblocks, and I was able to make progress that I believe will be beneficial in the future.

The first large feature that I did was the multi-axis paginator. This feature provides a new level of complexity to navigating our SEO data by merging crawl data with a combination of categorization in both ascending and descending directions. It was not easy, and it took a great deal of understanding to know how our current data structure worked with the pagination system. I spent a large portion of my day tinkering with several different algorithms for pagination, and in the end, I came up with one that will both increase our domain ceiling and help to improve our user experience when accessing SEO data. I built a system that can process a much larger dataset, which will be needed as we grow.

New features may also mean new bugs, and today was no exception. I found some problems with how the SEO scanner was displaying the domains. Namely, I found the system was trying to process rows that were marked as 'awaiting location', which was contributing to delays and cluttering the results. I made a fix so that these unresolved domains are excluded from the results. This change also improves the performance of the scanner and allows it to focus on actionable domains.

Besides fixing the location problem, I also increased the per-tick enrichment from 20 to 60. This will make the backfill process complete much quicker and is vital to keeping our SEO data accurate and useful. I saw that the original enrichment rate was far too low for the amount of data we had and by increasing it, I will cut down the time it takes to fill our database with important data.

Today, I made sure that the paginator and the domain filter were running in tandem. I had to change a lot about how the system worked to make multi-axis pagination work, which involved a lot of restructuring on our end. There were some points where I thought I would have to abandon the idea of a paginator entirely. I was very happy to find a solution that was aligned with my goals.

Today, I learned the value of testing your ideas before committing to them. I had to test the multi-axis paginator a million times to improve the user experience. It was a lot of work, but I felt that it was important to keep the larger scope in mind as I was building a one-person company that I hoped in the future would be worth a billion dollars.

I feel that my progress is boosted by how much work I did on the SEO scanner. I was so pleased to ship another feature. I am always glad to see how AI tools can improve a development environment without unnecessary and expensive collaborations. I've had to stay focused in order to achieve my goals and these tools have helped maintain my focus and productivity.

I’m excited as I finish this entry about the future of the SEO scanner and the role it plays in the mission as a whole. My improvements today mean a lot to our agentic SEO and set the stage for more improvements to come. I’m eager to see the effect of the changes on our performance and user satisfaction with the platform as we continue to develop it.