The feed didn't tell me which way the bus was going

Context

Bus View shows every bus in Sydney on a live map and turns the late ones into a short list of incidents. The data comes from Transport for NSW's open GTFS-realtime feeds: one for vehicle positions, one for trip updates (the delay per stop), both as protobuf messages refreshed every few seconds.

The first version decoded the feeds, put a dot per bus on the map, and grouped late buses by route and direction. It ran on the first evening. It was also wrong in two ways I didn't notice for a while, and both came from the same habit: trusting a field that was never there.

What I started noticing

Every single incident said outbound. Not most of them, all of them. And the delays looked far worse than the map: a school route reporting 47 minutes late at two in the afternoon, buses with zero delay sitting next to buses with twenty.

I first assumed the afternoon peak was just bad. Then I dumped one raw message and counted. Of about 1,700 vehicles, none carried a direction_id. The feed doesn't send it. Neither feed sends a headsign. The decoder had been filling the gap for me.

The decoder was protobufjs, and its toObject call returns proto defaults for absent fields. A missing direction_id became 0, which GTFS defines as outbound. A missing delay became 0, which my code read as "on time". A missing stop time became 0, which my code read as "already passed", so the delay was being taken from the wrong stop.

Three different bugs, one cause: I could not tell "zero" from "not sent".

The approach that worked better

The first fix is one option and one helper. Decode without defaults, and check presence before using a number.

// Absent fields come back as undefined, not 0.
const feed = transit_realtime.FeedMessage.toObject(msg, {
  longs: Number,
  defaults: false,
});

function hasNum(v: number | null | undefined): v is number {
  return typeof v === "number" && Number.isFinite(v);
}

const delaySec = hasNum(stop.departure?.delay) ? stop.departure.delay : null;

After that, direction was honest: unknown, for every bus. Which is correct and useless. A controller needs to know whether the late buses are heading into the city or out of it.

The direction lives in a different dataset. Transport for NSW publishes a static GTFS bundle, a zip of CSV files that describes the timetable: every trip, its route, its direction, its headsign, every stop, and the shape of every route. It's big, and it changes every few weeks, but the join is tiny: the realtime feed gives a trip_id, the timetable knows everything about that trip.

So a build script downloads the bundle, keeps only the columns the app needs, and writes them as small compressed JSON files. Trip to direction and headsign is about 300 KB. Stop to name, route to shape, and route to stop list bring the whole set to under four megabytes. The server loads them lazily and joins them into each vehicle at request time.

export function lookupTrip(tripId: string) {
  const rec = staticTrips()?.trips[tripId];
  if (!rec) return null;
  const [dir, headsignIndex] = rec;
  return {
    direction: dir === 0 ? "outbound" : dir === 1 ? "inbound" : "unknown",
    headsign: headsigns[headsignIndex] ?? null,
  };
}

Coverage on the first run was 100 %: every trip in the live feed existed in the timetable. The browser never sees either raw feed; it gets one vehicle object with a direction, a destination, and named next stops.

The join also fixed a third thing I hadn't planned for. Without a headsign, "buses going to the city" was a guess based on direction. With a headsign, it's a string match.

Why this mattered

• The direction grouping went from meaningless to the backbone of the incident list. Forty late buses on one road, split by direction, is two incidents with two different causes.
• The delay numbers became believable. Median 1.7 minutes late, 90th percentile 8 minutes, which matches how the network feels.
• The mistake was invisible at the UI level. Every screen looked fine; the map was simply lying. The only way I found it was by counting fields in a raw message.

Trade-offs

• The static bundle goes stale. A new timetable can rename trips, and a bus running a trip the build doesn't know about falls back to unknown. The app carries a data version so the browser refetches route shapes when the bundle changes, but rebuilding is still a manual step.
• Four megabytes of lookups is fine for a server that serves one city. It would not be fine in the browser, and it would not scale to a national feed without a proper store.
• defaults: false shifts the burden onto every reader of the data to handle undefined. That's the right burden, but it's a lot of hasNum calls.

What I would keep doing

Dump a raw message and count fields before writing any logic against a feed, especially a protobuf one, where the schema promises fields the publisher never fills. Treat "missing" as its own value all the way through the pipeline, never as zero. And assume a real-time feed is only the changing half of the data: the stable half lives somewhere else, and the join is usually smaller than you fear.