Differences

This shows you the differences between two versions of the page.

--- projects:software:malaysia_gtfs_opentripplanner_onebusaway [2024/12/31 12:27] – [OpenTripPlanner] Andrew Yong
+++ projects:software:malaysia_gtfs_opentripplanner_onebusaway [2025/01/01 07:25] (current) – placeholder for OneBusAway connection Andrew Yong
@@ Line 1: / Line 1: @@
 ====== Connecting Malaysia's Transport Data APIs to OpenTripPlanner and OneBusAway ======
-FIXME        This page is a work in progress.
+===== Problem Statement =====
+I recently moved to Kuala Lumpur, Malaysia and rely on public transportation as part of my daily commute.
+==== Wayfinding ====
+The Klang Valley has an extensive public transportation network. However, its planning is constrained physically and financially by the valley's existing road network sprawl, undulating topography and [[https://civil.utm.my/wp-content/uploads/2016/12/Land-Acquisition-for-Klang-Valley-Mass-Rapid-Transit-Line-1-Project.pdf|tedious land acquisition]], resulting in a large number of [[https://myrapid.com.my/bus-train/rapid-kl/rapid-kl-integrated-transit-map/|interchanges between rail lines]] and transit modes that are not immediately obvious. Often, the best route or easiest transfer is not always at a designated interchange or hub). Thus, **wayfinding** is a problem I face.
+==== Realtime Information ====
+The Klang Valley has [[https://www.planningmalaysia.org/index.php/pmj/article/download/Article%201-8/36|extensive suburban sprawl]], so the last-mile journey relies on the  [[https://myrapid.com.my/bus-train/rapid-kl/bus/|bus network]] (other than car ownership - park and ride). However, that sprawl, coupled with Klang Valley's notorious traffic congestion, makes it expensive and challenging (from my experience as a public bus captain, I would say, impossible) to maintain a consistent and short service interval on the bus network.
+Short of public transportation reform, I want to make best use of the level of service that is already there.
+For example, the loop service (service 650) that serves my taman (estate, literally "park") runs at a planned frequency of between 20 to 40 minutes, but due to congestion along Jalan Klang Lama (compounded by inclement weather), can have headways degraded to 40 to 60 minutes.
+With such long headways, the wait for the bus becomes half of, or more, of the commute time; rather than giving up on public transportation, having **realtime information** (specifically, realtime bus arrival times) means I can make more efficient use of my time by running errands or having a meal while waiting for the bus.
+===== Existing Solutions =====
+=== Maps ===
+  * [[https://myrapid.com.my/bus-train/rapid-kl/rapid-kl-integrated-transit-map/|RapidKL Integrated Transit Map]]
+=== Directions (Wayfinding), Schedules and Arrival Times ===
+  * [[https://www.google.com.my/maps|Google Maps]]
+    * Good:
+      * Wayfinding (directions)
+    * Bad:
+        * Live arrival times are not accurate
+        * Scheduled departure times are not accurate or missing entirely
+  * [[https://moovitapp.com/|Moovit]] ([[https://apps.apple.com/us/app/moovit-public-transit/id498477945|App Store]]/[[https://play.google.com/store/apps/details?id=com.tranzmate|Play Store]])
+    * Good:
+      * Live arrival times are accurate (±1-3 min)
+      * Scheduled departure times
+    * Bad:
+      * Constant and annoying full-screen advertisements (or monthly subscription)
+      * Live bus location is behind monthly subscription paywall
+  * [[https://myrapid.com.my/pulse/|MyRapid PULSE]] ([[https://apps.apple.com/my/app/myrapid-pulse/id6736607730|App Store]]/[[https://play.google.com/store/apps/details?id=com.prasarana.pulse.v4|Play Store]])
+    * Good:
+       * Live arrival times are accurate (±1-3 min)
+       * Scheduled departure times
+    * Bad:
+      * Wayfinding (directions)
+===== ... And a Few New Problems =====
+==== No Swiss Army Knife ====
+There is no //Swiss Army knife// app that does everything (wayfinding + schedules + arrival times) well without having some kind of catch.
+**A new problem arises that multiple apps/information/facilities above need to be used to find the best route.**
+==== Closed Source + Ads ====
+Closed Source is probably a weak criticism to make, as are ads, since the GTFS datasets are huge and processing them is computationally (and thus financially) expensive. I think these solutions should exist for the general public who are willing to tolerate advertisements or pay to have an ad-free experience, at least, until a better option arises. After all, developers need to eat and earn a living wage, too.
+But it is a problem to me because:
+  * Closed source means we cannot learn how the data is consumed, computed and presented, and philosophically I believe in freedom of knowledge rather than be fully dependent on black boxes built by corporations
+  * If I dislike ads and am willing to bear the cost of running my own infrastructure, then I want an ad-free self-hosted experience
+Furthermore, knowing how my solution works and bearing the cost of hosting it, I am incentivized to make it cost-effective rather than being incentivized to burden the user with ads or a subscription instead.
+===== Building a Solution =====
+Given the lack of a comprehensive solution, and fully at risk of [[https://xkcd.com/927/|creating yet another competing and imperfect solution]], I decided to explore building a solution.
+Even if I do not succeed (or get distracted by other commitments), at least the documentation below will serve as a starting point for anyone who finds themselves experiencing the same problems I did.
+:!: This is a **living document** and the documentation below will continue to evolve as I try/build/fix things :!:
 ===== Malaysia's Transport Data APIs =====
+Building a solution starts strong, because the [[https://data.gov.my/|Government of Malaysia Open Data Portal]] provides two authoritative public transportation data APIs in standard [[https://gtfs.org/|GTFS]] formats.
   * [[https://developer.data.gov.my/realtime-api/gtfs-static|GTFS Static API]] - consumed to build OpenTripPlanner Graph
   * [[https://developer.data.gov.my/realtime-api/gtfs-realtime|GTFS Realtime API]] - consumed by OpenTripPlanner Realtime Updaters
 ===== OpenTripPlanner =====
-  - Create a working directory for OpenTripPlanner, for this example, we will use ''./opentripplanner'':\\ ''mkdir opentripplanner && cd ./opentripplanner''
+> OpenTripPlanner (OTP) is an open source multi-modal trip planner, focusing on travel by scheduled public transportation in combination with bicycling, walking, and mobility services including bike share and ride hailing.
-  - Download latest GTFS static data (TODO: automate this - need to make the URLs pass OTP built in https fetcher validation)
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/mybas-johor%%''
+Instructions are written with [[https://docs.docker.com/compose/install/|Docker Compose]] in mind, but it's possible (with many more steps) to [[https://docs.opentripplanner.org/en/latest/Basic-Tutorial/|run OpenTripPlanner standalone]] with the configuration below.
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/ktmb%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-penang%%''
+**Demo: https://otp.ndoo.my/**
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-kuantan%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-mrtfeeder%%''
+==== Preparation - Files ====
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-rail-kl%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-kl%%''
+  - **Create a working directory for OpenTripPlanner**, for this example, we will use ''./opentripplanner'', you may wish to use different persistent storage e.g. for a Docker volume
-  - Download the latest OSM database for Malaysia(+):\\ ''%%wget https://download.geofabrik.de/asia/malaysia-singapore-brunei-latest.osm.pbf%%''
+    * ''mkdir opentripplanner && cd ./opentripplanner''
-  - Additional configuration is required because Malaysia’s GTFS files do not include timezone: FIXME TODO
+  - **Create configuration files**
-  - Build OpenTripPlanner graph:\\ ''%% docker run --rm -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --build --save%%''
+    - **Build Configuration** to download regional OpenStreepMap data and GTFS data (GTFS are not the documented API links [[https://github.com/opentripplanner/OpenTripPlanner/issues/6351|due to URI extension validation in OpenTripPlanner]])\\ <file json build-config.json>{
-  - Run OpenTripPlanner server
+  "osm": [
-    * Docker:\\ docker run -it --rm -p 8080:8080 -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --load --serve
+    {
-    * Docker Compose: TODO
+      "source": "https://download.geofabrik.de/asia/malaysia-singapore-brunei-latest.osm.pbf"
+    }
+  ],
+  "transitFeeds": [
+    {
+      "feedId": "ktmb",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/ktmb/gtfs_ktmb.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "mybas-johor",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/mybas-johor/gtfs_mybas.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-kl",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kl.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-kuantan",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kuantan.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-mrtfeeder",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_mrtfeeder.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-penang",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_penang.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-rail-kl",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_rail_kl.zip",
+      "type": "gtfs"
+    }
+  ],
+  "transitModelTimeZone": "Asia/Kuala_Lumpur"
+}
+</file>
+    - Router Configuration for Malaysia GTFS Realtime vehicle position feeds\\ <file json router-config.json>{
+  "updaters": [
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/mybas-johor",
+      "feedId": "mybas-johor"
+    },
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/ktmb",
+      "feedId": "ktmb"
+    },
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kl",
+      "feedId": "rapid-bus-kl"
+    },
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-mrtfeeder",
+      "feedId": "rapid-bus-mrtfeeder"
+    },
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kuantan",
+      "feedId": "rapid-bus-kuantan"
+    },
+    {
+      "type": "vehicle-positions",
+      "frequency": "PT30S",
+      "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-penang",
+      "feedId": "rapid-bus-penang"
+    }
+  ]
+}
+</file>
+==== Preparation - Docker Compose ====
+<file yaml compose.yaml>services:
+  opentripplanner:
+    command: --load --serve
+    environment:
+      - JAVA_TOOL_OPTIONS=-Xmx8G
+    expose:
+      - 8080
+    image: opentripplanner/opentripplanner:latest
+    restart: unless-stopped
+    volumes:
+      - /path/to/opentripplanner:/var/opentripplanner
+</file>
+==== Build Graph ====
+  * Docker Compose:\\ ''%%docker compose run --rm opentripplanner --build --save%%''
+  * Docker:\\ ''%% docker run --rm -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --build --save%%''
+==== Start OpenTripPlanner Server ====
+    * Docker Compose
+      * ''%%docker compose up -d opentripplanner%%''
+      * ''%%docker compose logs -f opentripplanner%%'' to view logs
+    * Docker (foreground): ''%%docker run -it --rm -p 8080:8080 -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --load --serve%%''
+FIXME unfinished writing, a conclusion and some screenshots here would be nice
+===== Connecting to OneBusAway =====
+We will use OneBusAway as a mobile app frontend to OneTripPlanner.
-FIXME FIXME FIXME FIXME FIXME
+FIXME unfinished writing, OneBusAway + OpenTripPlanner works on Android, but not iOS; there was [[https://github.com/OneBusAway/otpkit|a GSoC 2024 project to support OpenTripPlanner on OneBusAway iOS]] but the app implementation never happened.

ndoo.sg

User Tools

Site Tools

Differences

Page Tools