One of the great joys of working on a search engine is that you get to reverse engineer SEO spam, and overall study how it evolves over time. I’ve been noticing the search engine spam strategy of adding ‘reddit’ to page titles for a few years now, but it feels like it’s been growing a lot recently. I don’t think...

I get significantly more work done when I unplug my computer from the Internet. It’s not that my productive output drops to zero when I’m plugged in, but more like 70%. Despite many of the tools that I use requiring a connection, and certainly the Internet containing a wealth of information that might expedite my...

You have a hobby you’ve been into for a decade or more. You like talking about your hobby, and your friends and family, after listening to these things for as long as you’ve been into them, maybe aren’t as excited to always hear about it as you are about discussing them, so in an act of compassion you create a...

This is a bit of a retrospective of every project I’ve worked on, as far as I remember them. I’ve tried to unearth any artifacts that remain. Far from everything is flattering and resounding success, but then again, maybe that’s good. There are definitely patterns in the things that didn’t pan out. Earliest Traces I...

Marginalia Search very recently gained the ability to filter results by Autonomous System, not only searching by ASN but by the organization information for that AS. At a glance this seems like a somewhat frivolous feature, but it has interesting effects. Autonomous Systems are part of the Internet’s routing...

A simple guide to reading in 9 simple steps Navigate to the desired article. Dismiss the GDPR banner It may seem safe to start reading, but you need to wait about 10 seconds as the various ad auctions resolve and scripts load in Wait while the article is populated with ads. While the article is in front of you,...

The Marginalia Crawler has seen improvements! A long term problem with the crawler design is that if for whatever reason the crawler shuts down, then it needs to re-start fetching whatever domains it was currently traversing during the termination from zero. This isn’t fantastic, since not only does crawling a...

I’ve been working on getting anchor tag keywords into the search engine, basically using link texts to complement the keywords on a webpage. The problem I’m attempting to address is that many websites don’t really describe themselves particularly well. As Steve Ballmer’s stage performance once illustrated, merely...

So a bit of an update on what I’ve been working on. This will be adapted into release notes in a while, but I haven’t quite wrapped a bow on the change set yet. Still, it has certainly been a few weeks. Didn’t quite land how busy I’ve been until I set down to draft this post. Them’s some changes, and I’m skipping a...

As a general observation, I tend to be more productive when I know what to do next at any given moment. There are days when I’ve seemingly gotten a “week” of work done on an afternoon, those are the days when what I needed to do was very clear, and I basically just had a list of items to tick off one by one. There...

So the search engine is moving to a new server soon, thanks to the generous grant mentioned recently. If you visit search.marginalia.nu now, it may or may not use the old or new server. It’ll be like this for a while, since I need them both for testing and maintenance type work. I’ll also apologize if this post is a...

I ran into a bit of a puzzling situation yesterday, testing some of the new index construction changes before they’re going live in a few days. The process crashed with a pretty non-descript stack trace complaining about illegal instructions, so first glance it looked more like it was within the realm of freak JVM...