· 3.8k words
Making Victoria’s traffic signal data accessible.
The implementation decisions behind VicSignals: a tool for inspecting open traffic volume information at any signalised intersection in Victoria, Australia.
Most signalised intersections1 in Victoria use sensors to dynamically drive the behaviour of traffic lights. The sensors’ main purpose is to see when people are waiting at the stop line, but as a by-product they also keep an approximate count of the vehicles passing through each approach. The Department of Transport and Planning (DTP) publishes these counts as open data,2 with records for about 5,000 intersections and crossings going back to 2014.
This information is useful for transport planning because it provides a very granular view of what is happening on the road network. However, the raw data is large and awkward to work with. VicSignals is a web tool I built to make this resource easier to explore. With it, you can choose any signalised intersection on a map to see how traffic varies by time of day, day of week and season, and over the last twelve years of available records (including through COVID-19 and Melbourne’s rolling lockdowns).
This post explains how the tool works and why it was built the way it was. It covers the structure of the source data and its issues, how it is transformed into something more amenable to analysis, and side quests such as testing how well language models can read hand-drawn detector maps.
Inductive-loop sensors
Victoria’s traffic signals run on SCATS, the Sydney Coordinated Adaptive Traffic System.3 It adapts its timing to the traffic it senses, mostly through inductive loops made of wire laid into the road surface. The presence of metal–such as a car–above one of these loops changes its inductance which the controller senses and tallies as an activation.
As mentioned, the controller mainly wants to know whether anyone is waiting at an intersection approach. The count of activations is somewhat incidental, but it is these values that are released publicly: for every detector (a numbered loop) at every intersection and crossing, the dataset provides the total number of times they were triggered, binned into 15-minute blocks each day.
| Situation | Counted | |
|---|---|---|
| CarIts metal triggers the loop. | 1 | |
| Bumper to bumperThree cars read as one. | 1 | |
| MotorcycleLittle metal, easily missed. | 0 | |
| TruckIts trailer rides too high. | 2 | |
| Lane changeIt crosses both loops. | 2 | |
| Faulty loopCracked road, broken wire. | 2 |
How a loop behaves in various situations.
Because loops are designed primarily to detect presence rather than to count things, an activation is not always exactly one vehicle. Closely spaced vehicles in slow traffic can register as a single activation, a motorcycle may not have enough metal to register at all, and a truck with a high trailer or a car drifting across two lanes can be counted twice. Loops can also degrade as the road surface cracks and breaks the wires.
In ordinary conditions, we’d expect most activations to simply represent one vehicle passing over a loop. In any case, the same equipment is used in the same way across the whole network, so whilst counts shouldn’t be read as an exact number of vehicles, they are, at the very least, a consistent measure of activity that can be compared between places and over time.
What the data shows
Before exploring the structure of the raw data it’s worth looking at an example of how this all comes together to provide something useful. The visualisation below shows the activity at an arterial intersection in Melbourne’s eastern suburbs across ten detectors. Summing these gives the total activations for every 15 minutes of 2025, which can then be averaged for each day of the week.
2025Average activations in a day
The average Monday to Sunday at Canterbury Road and Rooks Road in 2025, public holidays aside. Marks under the year are school holidays.
The average Monday to Sunday at Canterbury Road and Rooks Road, four weeks at a time through 2025. Dashes are public holidays. Marks under the year are school holidays.The shape of a weekday is what we would expect – very little traffic overnight, a morning peak around 8:30am leading into a plateau throughout the middle of the day (the inter-peak), and finally a larger evening peak around 5pm.† Weekends have a different shape altogether, with a single broad rise that tops out around midday.
What changes throughout the year is mostly the overall level of traffic rather than the shape of the daily profiles. January weekdays are around 10 to 15% quieter than normal, and each school holiday period takes around a tenth of the activity away from the peaks. Public holidays are generally quieter than a typical Sunday. Regularity is what makes the data useful, because it lets us directly quantify changes across time and space.
Public does not mean accessible
DTP publishes three separate datasets, all of which are needed to interpret activity at any given signalised intersection:
- The activation counts themselves.
- A list of every signalised site in the state (an intersection or a crossing) with its coordinates.4
- A set of configuration sheets, sometimes referred to as op sheets, which include a drawing of each intersection showing where its detectors are located and how they are numbered.5
Without the last two the count data is uninterpretable. Altogether, this comes to 61 downloads and around 76 GB (at the time of writing), including the small list of sites and their coordinates.
76.1 GB to download61 files190 GB unzipped253,125 files
Everything DTP publishes about its traffic signals.
Counts by day rather than site
The counts are spread across yearly (and sometimes monthly) ZIP archives that expand to over 100 GB on disk. Inside is one CSV file for each day since 2014, with one row for each detector and a column for every 15-minute block in that day. Thus, one day holds around 120,000 rows and the entire dataset more than 420 million. If Excel is your tool of choice–which it is for most–you could probably open one of these files, but even a week’s worth would be pushing it.6
How those files are packaged has changed several times. Some years are a single ZIP archive with every day packed flat within it. Others are an archive of twelve nested archives, one for each month, and one year goes further by also compressing each day, forming a rare three-layer nest of ZIP archives.†
The larger issue is the shape of the data. One CSV file covers one day across every site, whereas people are more likely to ask about one site across many days. To understand activity at a single intersection through 2025, for instance, you would need to download 1.6 GB and read every row of every file, only to discard most of it.
The contents of the files have their own problems, most of which are difficult to find without a detailed audit: rows that appear twice, rows dated differently to the file they sit in, files where the columns are shifted over halfway through. Daylight savings is another example, where the treatment of the clock changes has varied from year to year.
Time that passed6 April 202513 April
Quarter hours in the file
Time that passed5 October 202512 October
Quarter hours in the file
How the clock changes in 2025 are represented in the count data.
In April 2025, the repeated hour holds around 1.7× the activations of the same hour a week later, which is what we would expect if both passes had simply been added together. In October, the missing hour is filled with a status code instead. Other years behave differently. This is unlikely to make a difference to any analysis, but it’s important to know when the dataset is behaving oddly.
Maps buried in documents
The op sheets present their own challenge. These are 27 ZIP archives totalling 57 GB, with one document for each site. A little under half are scanned PDFs–some filled in by hand–which account for nearly all of that size. The rest are Excel workbooks saved as web pages, each with their own folder of images, which is how the archives come to hold around 250,000 files.
In either form, what we need is a single drawing somewhere in the document–the detector map–which shows the lane each detector sits in. Finding out what any one detector measures means downloading the right archive, finding the right document inside it, and consulting the right picture.
Maidstone Street, Altona3484.pdf, 19 MB, 10 pages
Canning Street and Military Road2716.pdf, 27 MB, 13 pages
Canterbury Road and Rooks Road0865.htm and 33 more files, 0.8 MB, 13 sheets
Example op sheets and where the detector map is located in each.
A file for each site
The counts need to be smaller, but more importantly they need to be randomly accessible by site. In the raw data, the history of one site is scattered across thousands of daily files and has to be reconstructed from all of them before any analysis can begin. To address both problems, every archive was read once and the activations rewritten into a custom binary format, with one file generated for each site and calendar year.
By day, as downloaded19.3 GB in 33 archives
By site, as stored9.5 GB in 55,026 files
| Header | 15 B | Site, year and days |
|---|---|---|
| Detectors | 96 B | 24 numbers, 10 counting |
| Low bytes | 839 KB | By detector, time of day, then day |
| High bytes | 839 KB | 99.7% zero |
The counts as downloaded compared with their new binary format.
Much of the original data is actually zeros. Of all the counts released since 2014, 61% are zero, largely because many sites are assigned either 24 or 32 detectors even though in reality they have only four to eight. The CSVs write out every one of these zeros regardless.
The new storage format is intentionally simple. Each count is stored not as text but as a 16-bit number, which comfortably holds the largest observed value whilst leaving room to carry status codes and blank values unchanged.7 The counts are ordered so that a detector’s 8am on one day sits next to its 8am on the next, and the two bytes of each count are kept in separate halves of the file. As 99.9% of all counts are below 256, the second half is almost entirely zeros. This arrangement gathers similar values and zeros into long runs, which a general-purpose compressor8 can then collapse very effectively.
The result of this is a file for each site and calendar year that is small enough to serve directly to the browser. For example, all of 2025 at Canterbury Road and Rooks Road is 223 KB, whereas previously it was spread across 1.6 GB of archived data.
Analysis in the browser
With each site’s year of activations stored in a few hundred kilobytes, the browser can hold the counts itself and analysis can happen locally. A more conventional setup might involve storing the counts on a server as Parquet files, with a database such as DuckDB answering each query. This is a good place to start because it’s quite fast for relatively little effort. A common request might be calculating the number of activations between 7am and 9am at one site on each day of a year. Run against Parquet files holding the entire dataset, DuckDB can answer this in around 5 ms.
However, an answer produced on a server still has to physically travel. The request is sent from the client, the server calculates the answer, and the answer is then sent back to the client. From a home connection in Melbourne, when asking a data centre in the same city for a short answer the round trip alone takes approximately 7 ms before any actual work is done.9 A server written specifically for this one query (with the site’s counts already in memory) takes 0.6 ms, but still cannot answer any sooner than the round trip allows. JavaScript working through a site file the browser already holds takes 0.4 ms, and is finished before a request would have reached the server.
DuckDB over Parquet5.1 ms compute + 6.6 ms network
Purpose-built server0.6 ms compute + 6.6 ms network
Browser, not cached5.5 ms compute + 23.2 ms network
Browser, cached0.4 ms compute
Time to answer one query (muted bars represent network time).
The browser’s worst case is the first request for a given site when the file is not yet cached. Retrieving this file is slower than if a server provided the answer directly and in the example above it takes 23 ms to download.10 Decompressing and decoding it takes around 5 ms more, widening the gap to the server further.
This cost is paid once, though, and the cached case is by far the more important situation to optimise for. A site’s page in VicSignals is produced not by a single query but by many – the calendar, trend, daily profile and distribution views are each generated from the same counts, and all of them are regenerated whenever the user changes any of the filters. Someone exploring a site changes these constantly, and if each query were a request to the server, every single one would have to wait on the network and the page would be noticeably slower to respond. When the file is cached, every query can be answered without a network request at all.
Extracting detector maps
The detector map is the only part of an op sheet needed to interpret the counts, and they only take up a small part of each document. A scanned document averages around 30 MB,† so serving the documents as they are would mean sending a huge payload for the sake of one small drawing. At the same time, cutting out each detector map by hand from 5,000 op sheets is impractical (for me). Instead, a process was devised to mechanically locate the detector map in each document and crop it out.
Each page of each scanned op sheet was passed through text recognition.11 Only lines that started in the left 40% of the page or in its bottom 15% were read, which is where the diagram headings and footers are typically found. In almost every case, the map is the region from a line of text reading DETECTOR MAP down to the next heading or the footer. Older documents rule a box around the map, in which case the crop was taken along its borders. Otherwise, the detector map was trimmed to the smallest footprint containing the diagram’s ink.
Op sheets in Excel workbook form needed no text recognition. Instead, the heading was found programmatically in the markup of the file and the area beneath it was trimmed to the ink in the same way to retrieve the detector map.
Maidstone Street, Altona3484.pdf, 19 MB
Map, 24 KB
Canning Street and Military Road2716.pdf, 27 MB
Map, 37 KB
Canterbury Road and Rooks Road0865.htm, 0.8 MB
Map, 3 KB
Every document56.8 GB in 27 archives
4,841 maps, 102 MB34 without
How a detector map is found in an op sheet.
Together, the maps come to 102 MB, or about 21 KB each, whereas the archives they came from total 57 GB compressed. This represents a storage saving of 99.8% whilst also removing the need for the user to open a full op sheet alongside the tool.
Inferring detector approaches
A detector map shows which detector is which, but most analysis first needs the detectors gathered into approaches: these four count traffic heading north, these three the side street, and so on. The tool allows you to do this by hand, but it can be tedious for complex intersections and has to be repeated for each site. Even a rough starting point can be useful here, so suggested groups were generated in advance for every site.
Producing these suggestions meant properly interpreting the drawings which is a computer vision problem. The drawings also vary enormously in quality – some were drafted cleanly on a computer, whilst others were drawn by hand or saved at a resolution too low to read their features.
Dohertys Road and Morris RoadNo north arrow, a slip lane in every corner, and each road named once across both of its arms
Ballarat Road and Station RoadMany detector numbers are too coarse to read
Frankston-Dandenong Road and Abbotts RoadA poor copy of a construction plan, with detectors among its notes
Footscray Road and Pearl River RoadRamps, a slip road and two separate approaches from one side
Example detector maps that are difficult to read.
One of the core difficulties is simply working out which way is north. Maps are seemingly drawn at whatever angle suits the page and few carry a distinct north arrow. In some cases, the orientation is actually impossible to tell from the map alone – a road is often named only once across both of its arms, and a symmetrical crossroads can look the same no matter which way the map is rotated.†
Each map was fed into a frontier language model12 along with the site’s detector numbers. The model was asked only for what was visible on the drawing – the direction each arm of the intersection points on the page, any road name written on it, any north arrow or slip lane, and the arm each detector sits on. It was never asked for a compass direction.13 North was instead inferred afterwards by rotating the drawing over the roads as they are mapped in OpenStreetMap until its arms lay along them.
A drawing of a crossroads can be fitted to its roads in four ways. Road names usually rule out two of these, but the two that remain are opposites, and choosing incorrectly would name every northbound detector southbound. Something else on the drawing has to separate them, such as a north arrow or a slip lane that only one corner has.
| Top of drawing | Roads | Names | Slip lane |
|---|---|---|---|
| North | ✓ | ✗ | ✗ |
| West | ✓ | ✓ | ✗ |
| South | ✓ | ✗ | ✗ |
| East | ✓ | ✓ | ✓ |
A detector map rotated over the mapped roads until everything agrees.
Where nothing on the drawing separates the candidates, the best-fitting rotation cannot be trusted. With names and arrows hidden, it was right on only 37 of 84 test maps. Groups at these sites are left without a direction and are named by where they sit on the drawing instead, such as Top arm.
To understand how far the results could be relied upon, every map was read twice independently, and a third time where the first two readings disagreed. The two readings gave the same direction to 99.3% of the detectors that both had named. A group was given high confidence when both readings agreed on its direction and the drawing fitted the roads firmly, and medium confidence when the fit was loose or a further reading was needed to settle it.
51,757 detectors4,534 detector maps
- 26.7% Direction, high confidence
- 14.0% Direction, medium confidence
- 34.9% Arm only
- 24.4% Not placed
How well each detector was allocated in a suggested group.
Around two fifths of detectors ended up in a group with a direction, a third were grouped by arm alone and a quarter could not be placed at all. This is very much a work in progress and expensive to experiment with. One pass over every map is about 9,500 readings, which takes around six hours and US$450 at API prices, so each change to the method is slow to test.
Patterns across the network
Every example so far has focused on a single site. Taken across every site at once, the same counts show broader patterns, such as how activity fell during COVID-19 and how much of it has since returned.
Comparing one year with another takes some care, because the road network is modified over time and a site’s total activations can change without its traffic changing at all. To account for this, only the 3,784 sites with consistent records through 2019 were kept, and each was compared with itself as a share of its typical 2019 day for the same day of the week. Sites were then grouped into rings by their distance from Melbourne’s General Post Office (GPO).
2025Midweek activations as a share of 2019
Midweek activations during school terms as a share of 2019, by distance from Melbourne’s GPO.
During the lockdowns, traffic fell most sharply close to the city centre, where a typical midweek day in 2020 saw just under two thirds of its 2019 activity. Activity there has not fully recovered. In 2025, sites within 2 km of the GPO were still 10% below their 2019 levels and those between 2 and 10 km were 3 to 5% below. Sites further out than this have returned to their 2019 levels or only just surpassed them.
Daily profiles have changed as well, again depending on distance from the city centre.
0.1 to 2.4 km from the GPO
18 to 21 km from the GPO
71 to 476 km from the GPO
The day in 2025 against 2019 at three distances from Melbourne’s GPO.
The day in 2025 against 2019, moving outward from Melbourne’s GPO.Close to the GPO, every part of every day is quieter than it was in 2019, especially in the early morning and the evening. Friday mornings stand out – between 7am and 9am, a Friday is at only 82% of its 2019 activations, compared with 87% midweek.
Further out, the middle of the day recovers first. By around 10 km, weekdays between 10am and 3pm are back at their 2019 levels, whilst the early morning and the evening remain below them. Beyond 70 km, most of the day is busier than it was, and Sunday mornings are 10 to 13% above 2019. Overall, traffic has moved outward and towards the middle of the day.
The most likely explanation is the shift towards working from home. Fewer people commute to the city, fewest of all on a Friday, and those at home still make local trips through the middle of the day. Population growth on Melbourne’s fringe and in regional Victoria probably accounts for the rest.
Looking ahead
I hope VicSignals is useful to anyone curious about how traffic moves around Victoria, and I intend to keep it up to date as new data is released. It is an independent project and is not associated with DTP or anyone else. That said, it exists only because the Department publishes this data openly, and there is plenty more in it to find.
-
Signalised intersections are those with traffic lights or some other form of automated coordination. ↩︎
-
SCATS was developed in Sydney in the 1970s and now runs signals in many cities around the world. ↩︎
-
The evening peak is often more drawn out than the morning one. Traffic starts building for school pick-up from 3pm, and commuters head home from around 5pm, whereas the school run and the commute overlap in a shorter window in the morning. ↩︎
-
An Excel worksheet can hold 1,048,576 rows, but performance can degrade to unusable levels well before reaching this limit. ↩︎
-
It is probably common knowledge, but compressing an archive that is already compressed gains next to nothing. Compression works by removing repetition, and the first pass leaves almost none for a second to find, so each extra layer only adds the overhead of its own headers. ↩︎
-
A negative value in the source (3% of all values) is a status code rather than a count. A blank value is kept distinct from a zero. ↩︎
-
This is Brotli, which every current browser can decompress natively. ↩︎
-
Of that, almost 5 ms is the bare round trip of one packet out and one back. To a data centre in Sydney, the round trip is 17 ms. ↩︎
-
This is obviously subject to the speed of your internet connection. ↩︎
-
Some op sheet PDFs exceed 100 MB. ↩︎
-
A person who knows the site could often still settle it from a detail such as where the signal controller is drawn, but to do this at scale would require high-resolution satellite imagery. ↩︎
-
Anthropic’s Claude Opus 5.5. ↩︎
-
When the model was asked for compass directions directly, the ones it was unsure of reversed from one run to the next on the same drawing. ↩︎