Led by Keolis Downer, the proud operator of Yarra Trams, the Smarter Trams Program is revolutionising the world’s largest tram network.
Yarra Trams operates the world's largest tram network, accommodating over 200 million trips annually. It's a complex network comprising over 500 trams, 73% of which share their routes with cars, buses, and cyclists. Such interactions can cause delays or disruptions to the planned tram journey times.
The Smarter Trams Program implements real-time and dead-reckoning GPS on 399 trams, real-time Automatic Passenger Counting on 340 trams, and CCTV on 59 trams. The data collected will significantly improve passenger information, timetable planning, and safety reviews. It also has the potential to trigger traffic lights to reduce road congestion, as per the Government's Smarter Roads initiative.
There have been notable challenges in accessing real-time data about tram crowdedness and reliable tram locations within the Victorian tram network. However, recent advancements in technology have presented new opportunities for employing motion sensor technology and GPS sensors to improve the availability, accuracy, and reliability of data.
Portable was engaged to work on the development of a back-end system which would enable the transit and transformation of data captured, with the goal of enabling real-time data analytics that supports passenger counting and traffic light override capabilities. Focussing on the A, B, and C class trams, the outcomes of this work would pave the way for D and E Class trams to provide the same data insights in the future.
Data sent to the Department of Transport & Planning Victoria and Yarra Trams allows for automated real-time tram network optimisation decisions. For instance, if a tram is running late and is crowded, the system can ensure it gets more green lights. Conversely, if it is on time or not crowded, it can follow the standard cadence of traffic lights.
Using real-time data can also help identify areas of congestion, such as specific routes, dates, and times. Yarra Trams can then investigate and optimise these network bottlenecks. This innovative technology utilises real-time data to enhance the effectiveness and customer satisfaction of Victoria's tram network.
The project's main challenges were associated with managing the massive volume of data generated and the short time frame allowed to ensure the data could work with the traffic light priority system.
It was clear that Yarra Trams would need a technical design optimised for performance while handling large amounts of data and so a set of requirements were defined:
- Scalable data management — The system generates 200 pieces of data per second, a number that is increasing daily.
- Low data latency — Traffic light priority systems require data to be communicated in under 6 seconds to be effective.
- Effective automation — The platform and processes need to be fully redundant and capable of automatic recovery.
Portable provided Project Management, Business Analysis, Solution Architecture, and Engineering support for the setup of a data system. The source of the new data was the rollout of sensor-based passenger counter hardware. Portable also supported the modification of existing on-vehicle systems to allow for GPS-based geolocation of the trams, displayed via a visual interface.
The project phases are outlined below.
Technical Design Phase
Yarra Trams received GPS data from units installed in the trams, while passenger count data came from real-time sensors on the trams. The system then collated and merged this data, along with contextual scheduling data, such as whether the tram was on time and its expected arrival & departure times.
A key design decision was to pass the data through the Event Hub component offered by Azure. This allowed multiple stakeholders to subscribe to the data and provided a secure, scalable solution.
Data was then sent to various stakeholders, such as the Department of Transport & Planning Victoria, YarraTrams' tramTRACKER App, and YarraTrams' internal Data Lake. Each of these entities utilised the data for specific purposes, like traffic light control decisions, accurate tram arrival predictions, and data archiving and analysis, respectively.
Data Setup for Performance
The project requirement was for total transit time from tram to the Department of Transport & Planning Victoria to be under 5 seconds for thousands of messages per second.
Portable worked alongside Yarra Trams stakeholders to develop a solution architecture service to optimise capacity and production load. Our technical solution introduced horizontal scaling, which evenly balanced the data load across multiple servers in an automatic, transparent, and cost-effective manner.
Data Setup for Redundancy
The entire system had to be fully and automatically redundant. If an error occurred, the system needed to adjust itself to work as if nothing had failed, while notifying support staff about the non-functional server.
In the process of initially engineering the system for performance, we enabled the spin-up of additional server apps to handle throughput for potentially thousands of trams in the future. Interestingly, this allowed us to achieve redundancy without additional work. If one server fails, the remaining servers automatically rebalance the load, and the whole process functions as if nothing happened.
To assist with this, we designed a monitoring and alerting system that automatically opens and escalates cases in ServiceNow, with options for SMS and/or push notifications for severe incidents.
A significant part of the testing phase involved exploring edge cases to ensure a high level of accuracy.
The Portable team completed functional and load tests for 240-500 trams and undertook extensive security testing and management of the deployment pipeline and processes.
With new data available for the first time, an opportunity emerged to improve data visualisation.
Yarra Trams, our partner Dear Watson, and Portable worked together to roll out the primary visualisation platform in Microsoft Power BI. We also set up a pipeline that synthesised data from a data mart to feed the Power BI platform.
This analytics platform enable the Yarra Trams team to gain real-time insights for network planning.
These insights include:
- monitoring tram and route uptime
- identifying times and geographical locations of network inefficiencies and congestion
- and understanding passenger data attributes, trends, and patterns.
This information allows Yarra Trams to prioritise optimising tram routes in specific areas and times of the day that are inefficient and potentially disrupting other trams.
The Portable Team thoroughly enjoyed the opportunity to work on this challenging data project to ensure high performance and data quality while providing real-time crowding data and GPS location data as a critical function of the Smarter Trams Program.
This new system will pave the way for numerous customer improvements in the near future including:
1. Allowing customers to see how crowded a tram is before it arrives and make an informed decision about planning their journey.
2. Enabling customers to see where the tram is currently located, through real-time GPS locations, which means transparency on tram arrival times, altered trips, or shortened routes.
3. Giving external organisations such as the Department of Transport & Planning Victoria the opportunity to use real-time data to inform traffic light prioritisation.
4. Allowing the Yarra Trams team to internally optimise points of congestion, inefficiencies, oversupply, and under-demand.
“The Smarter Trams Program will provide a capability that Yarra Trams has never had before. This foundation allows YT to send or receive real-time data on-tram in real-time.
The platform built will create opportunities for all manner of applications that require Vehicle to Vehicle, Vehicle to Infrastructure and Vehicle to Pedestrian/Passenger communications to occur. Traffic light priority is just the beginning.
Portable is a key partner in helping execute Keolis Downer to execute on its strategy to enable mobility for Victorians through data”.
Charles Valladares, Manager Strategy & Relationships — Yarra Trams
“Smarter Trams system is a way for Yarra Trams to crunch a huge amount of data, convert it into useful insights, and then take action from that data.
This was a challenging project but one which was executed well and which is sure to see Victoria retain its role as the global leader of suburban short-transit transportation.”
James Zhang, Technology Lead — Portable
- James Zhang, Technology Lead
- Anthony Daff, Technology Lead
- Caitlin Wintaur, Senior Producer
- Cam Stark, Developer
- Phil Giles, Developer
- Paul Thomas, Developer
- Jesse Day, Developer
- Terry Collins-Hawkins, Senior Developer