Site Tools


Sidebar

Work

Projects Travel Singapore Socials (External)

projects:software:malaysia_gtfs_opentripplanner_onebusaway

This is an old revision of the document!


Connecting Malaysia's Transport Data APIs to OpenTripPlanner and OneBusAway

Problem Statement

I recently moved to Kuala Lumpur, Malaysia and rely on public transportation as part of my daily commute.

Wayfinding

The Klang Valley has an extensive public transportation network. However, its planning is constrained physically and financially by the valley's existing road network sprawl, undulating topography and tedious land acquisition, resulting in a large number of interchanges between rail lines and transit modes that are not immediately obvious. Often, the best route or easiest transfer is not always at a designated interchange or hub). Thus, wayfinding is a problem I face.

Realtime Information

The Klang Valley has extensive suburban sprawl, so the bus network (both trunk and feeder services) is a critical last mile component to public transportation network access (other than car ownership - park and rides). However, the sprawl, and the traffic congestion that the Klang Valley is notorious for, makes it expensive and challenging (or perhaps impossible, without major ground transportation reform) to maintain a consistent and short service interval on the bus network.

I don't plan to work on ground transportation reform, of course, but to make the best use of the level of service that is already there. The loop service (service 650) that serves my taman (estate, literally “park”) runs at a planned frequency of between 20 to 40 minutes, but due to congestion along Jalan Klang Lama (compounded by inclement weather), can have headways degraded to 40 to 60 minutes.

With such long headways, a problem arises that the wait for the bus becomes half of, or more, of the commute time; rather than giving up on public transportation, having realtime information (specifically, realtime bus arrival times) means I can make more efficient use of my time by running errands or having a meal while waiting for the bus.

Existing Solutions

Maps

Directions (Wayfinding), Schedules and Arrival Times

    • Good:
      • Wayfinding (directions)
    • Bad:
      • Live arrival times are not accurate
      • Scheduled departure times are not accurate or missing entirely
    • Good:
      • Live arrival times are accurate (±1-3 min)
      • Scheduled departure times
    • Bad:
      • Constant and annoying full-screen advertisements (or monthly subscription)
      • Live bus location is behind monthly subscription paywall
    • Good:
      • Live arrival times are accurate (±1-3 min)
      • Scheduled departure times
    • Bad:
      • Wayfinding (directions)

... And a Few New Problems

No Swiss Army Knife

There is no Swiss Army knife app that does everything (wayfinding + schedules + arrival times) well without having some kind of catch.

A new problem arises that multiple apps/information/facilities above need to be used to find the best route.

Closed Source + Ads

Closed Source is probably a weak criticism to make, as are ads, since the GTFS datasets are huge and processing them is computationally (and thus financially) expensive. I think these solutions should exist for the general public who are willing to tolerate advertisements or pay to have an ad-free experience, at least, until a better option arises. After all, developers need to eat and earn a living wage, too.

But it is a problem to me because:

  • Closed source means we cannot learn how the data is consumed, computed and presented, and philosophically I believe in freedom of knowledge rather than be fully dependent on black boxes built by corporations
  • If I dislike ads and am willing to bear the cost of running my own infrastructure, then I want an ad-free self-hosted experience

Furthermore, knowing how my solution works and bearing the cost of hosting it, I am incentivized to make it cost-effective rather than being incentivized to burden the user with ads or a subscription instead.

Building a Solution

Given the lack of a comprehensive solution, and fully at risk of creating yet another competing and imperfect solution, I decided to explore building a solution.

Even if I do not succeed (or get distracted by other commitments), at least the documentation below will serve as a starting point for anyone who finds themselves experiencing the same problems I did.

:!: This is a living document and the documentation below will continue to evolve as I try/build/fix things :!:

Malaysia's Transport Data APIs

Building a solution starts strong, because the Government of Malaysia Open Data Portal provides two authoritative public transportation data APIs in standard GTFS formats.

OpenTripPlanner

OpenTripPlanner (OTP) is an open source multi-modal trip planner, focusing on travel by scheduled public transportation in combination with bicycling, walking, and mobility services including bike share and ride hailing.

Instructions are written with Docker Compose in mind, but it's possible (with many more steps) to run OpenTripPlanner standalone with the configuration below.

Demo: https://otp.ndoo.my/

Preparation - Files

  1. Create a working directory for OpenTripPlanner, for this example, we will use ./opentripplanner, you may wish to use different persistent storage e.g. for a Docker volume
    • mkdir opentripplanner && cd ./opentripplanner
  2. Create configuration files
    1. Build Configuration to download regional OpenStreepMap data and GTFS data (GTFS are not the documented API links due to URI extension validation in OpenTripPlanner)
      build-config.json
      {
        "osm": [
          {
            "source": "https://download.geofabrik.de/asia/malaysia-singapore-brunei-latest.osm.pbf"
          }
        ],
        "transitFeeds": [
          {
            "feedId": "ktmb",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/ktmb/gtfs_ktmb.zip",
            "type": "gtfs"
          },
          {
            "feedId": "mybas-johor",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/mybas-johor/gtfs_mybas.zip",
            "type": "gtfs"
          },
          {
            "feedId": "rapid-bus-kl",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kl.zip",
            "type": "gtfs"
          },
          {
            "feedId": "rapid-bus-kuantan",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kuantan.zip",
            "type": "gtfs"
          },
          {
            "feedId": "rapid-bus-mrtfeeder",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_mrtfeeder.zip",
            "type": "gtfs"
          },
          {
            "feedId": "rapid-bus-penang",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_penang.zip",
            "type": "gtfs"
          },
          {
            "feedId": "rapid-rail-kl",
            "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_rail_kl.zip",
            "type": "gtfs"
          }
        ],
        "transitModelTimeZone": "Asia/Kuala_Lumpur"
      }
    2. Router Configuration for Malaysia GTFS Realtime vehicle position feeds
      router-config.json
      {
        "updaters": [
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/mybas-johor",
            "feedId": "mybas-johor"
          },
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/ktmb",
            "feedId": "ktmb"
          },
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kl",
            "feedId": "rapid-bus-kl"
          },
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-mrtfeeder",
            "feedId": "rapid-bus-mrtfeeder"
          },
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kuantan",
            "feedId": "rapid-bus-kuantan"
          },
          {
            "type": "vehicle-positions",
            "frequency": "PT30S",
            "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-penang",
            "feedId": "rapid-bus-penang"
          }
        ]
      }

Preparation - Docker Compose

compose.yaml
services:
  opentripplanner:
    command: --load --serve
    environment:
      - JAVA_TOOL_OPTIONS=-Xmx8G
    expose:
      - 8080
    image: opentripplanner/opentripplanner:latest
    restart: unless-stopped
    volumes:
      - /path/to/opentripplanner:/var/opentripplanner

Build Graph

  • Docker Compose:
    docker compose run --rm opentripplanner --build --save
  • Docker:
    docker run --rm -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --build --save

Start OpenTripPlanner Server

  • Docker Compose
    • docker compose up -d opentripplanner
    • docker compose logs -f opentripplanner to view logs
  • Docker (foreground): docker run -it --rm -p 8080:8080 -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --load --serve
projects/software/malaysia_gtfs_opentripplanner_onebusaway.1735715520.txt.gz · Last modified: 2025/01/01 07:12 by Andrew Yong