Differences

This shows you the differences between two versions of the page.

--- projects:software:malaysia_gtfs_opentripplanner_onebusaway [2024/12/31 18:42] – flesh out config files and docker compose file Andrew Yong
+++ projects:software:malaysia_gtfs_opentripplanner_onebusaway [2025/05/01 08:20] (current) – [OpenTripPlanner] kill demo to save resources Andrew Yong
@@ Line 1: / Line 1: @@
 ====== Connecting Malaysia's Transport Data APIs to OpenTripPlanner and OneBusAway ======
-FIXME This page is a work in progress.
+===== Problem Statement =====
-Instructions below are written with Docker and Docker Compose in mind; it is possible to run OpenTripPlanner standalone but that is left as an exercise to the reader.
+I recently moved to Kuala Lumpur, Malaysia and rely on public transportation as part of my daily commute.
+==== Wayfinding ====
+The Klang Valley has an extensive public transportation network. However, its planning is constrained physically and financially by the valley's existing road network sprawl, undulating topography and [[https://civil.utm.my/wp-content/uploads/2016/12/Land-Acquisition-for-Klang-Valley-Mass-Rapid-Transit-Line-1-Project.pdf|tedious land acquisition]], resulting in a large number of [[https://myrapid.com.my/bus-train/rapid-kl/rapid-kl-integrated-transit-map/|interchanges between rail lines]] and transit modes that are not immediately obvious. Often, the best route or easiest transfer is not always at a designated interchange or hub). Thus, **wayfinding** is a problem I face.
+==== Realtime Information ====
+The Klang Valley has [[https://www.planningmalaysia.org/index.php/pmj/article/download/Article%201-8/36|extensive suburban sprawl]], so the last-mile journey relies on the  [[https://myrapid.com.my/bus-train/rapid-kl/bus/|bus network]] (other than car ownership - park and ride). However, that sprawl, coupled with Klang Valley's notorious traffic congestion, makes it expensive and challenging (from my experience as a public bus captain, I would say, impossible) to maintain a consistent and short service interval on the bus network.
+Short of public transportation reform, I want to make best use of the level of service that is already there.
+For example, the loop service (service 650) that serves my taman (estate, literally "park") runs at a planned frequency of between 20 to 40 minutes, but due to congestion along Jalan Klang Lama (compounded by inclement weather), can have headways degraded to 40 to 60 minutes.
+With such long headways, the wait for the bus becomes half of, or more, of the commute time; rather than giving up on public transportation, having **realtime information** (specifically, realtime bus arrival times) means I can make more efficient use of my time by running errands or having a meal while waiting for the bus.
+===== Existing Solutions =====
+=== Maps ===
+  * [[https://myrapid.com.my/bus-train/rapid-kl/rapid-kl-integrated-transit-map/|RapidKL Integrated Transit Map]]
+=== Directions (Wayfinding), Schedules and Arrival Times ===
+  * [[https://www.google.com.my/maps|Google Maps]]
+    * Good:
+      * Wayfinding (directions)
+    * Bad:
+        * Live arrival times are not accurate
+        * Scheduled departure times are not accurate or missing entirely
+  * [[https://moovitapp.com/|Moovit]] ([[https://apps.apple.com/us/app/moovit-public-transit/id498477945|App Store]]/[[https://play.google.com/store/apps/details?id=com.tranzmate|Play Store]])
+    * Good:
+      * Live arrival times are accurate (±1-3 min)
+      * Scheduled departure times
+    * Bad:
+      * Constant and annoying full-screen advertisements (or monthly subscription)
+      * Live bus location is behind monthly subscription paywall
+  * [[https://myrapid.com.my/pulse/|MyRapid PULSE]] ([[https://apps.apple.com/my/app/myrapid-pulse/id6736607730|App Store]]/[[https://play.google.com/store/apps/details?id=com.prasarana.pulse.v4|Play Store]])
+    * Good:
+       * Live arrival times are accurate (±1-3 min)
+       * Scheduled departure times
+    * Bad:
+      * Wayfinding (directions)
+===== ... And a Few New Problems =====
+==== No Swiss Army Knife ====
+There is no //Swiss Army knife// app that does everything (wayfinding + schedules + arrival times) well without having some kind of catch.
+**A new problem arises that multiple apps/information/facilities above need to be used to find the best route.**
+==== Closed Source + Ads ====
+Closed Source is probably a weak criticism to make, as are ads, since the GTFS datasets are huge and processing them is computationally (and thus financially) expensive. I think these solutions should exist for the general public who are willing to tolerate advertisements or pay to have an ad-free experience, at least, until a better option arises. After all, developers need to eat and earn a living wage, too.
+But it is a problem to me because:
+  * Closed source means we cannot learn how the data is consumed, computed and presented, and philosophically I believe in freedom of knowledge rather than be fully dependent on black boxes built by corporations
+  * If I dislike ads and am willing to bear the cost of running my own infrastructure, then I want an ad-free self-hosted experience
+Furthermore, knowing how my solution works and bearing the cost of hosting it, I am incentivized to make it cost-effective rather than being incentivized to burden the user with ads or a subscription instead.
+===== Building a Solution =====
+Given the lack of a comprehensive solution, and fully at risk of [[https://xkcd.com/927/|creating yet another competing and imperfect solution]], I decided to explore building a solution.
+Even if I do not succeed (or get distracted by other commitments), at least the documentation below will serve as a starting point for anyone who finds themselves experiencing the same problems I did.
+:!: This is a **living document** and the documentation below will continue to evolve as I try/build/fix things :!:
 ===== Malaysia's Transport Data APIs =====
+Building a solution starts strong, because the [[https://data.gov.my/|Government of Malaysia Open Data Portal]] provides two authoritative public transportation data APIs in standard [[https://gtfs.org/|GTFS]] formats.
   * [[https://developer.data.gov.my/realtime-api/gtfs-static|GTFS Static API]] - consumed to build OpenTripPlanner Graph
   * [[https://developer.data.gov.my/realtime-api/gtfs-realtime|GTFS Realtime API]] - consumed by OpenTripPlanner Realtime Updaters
 ===== OpenTripPlanner =====
+> OpenTripPlanner (OTP) is an open source multi-modal trip planner, focusing on travel by scheduled public transportation in combination with bicycling, walking, and mobility services including bike share and ride hailing.
+Instructions are written with [[https://docs.docker.com/compose/install/|Docker Compose]] in mind, but it's possible (with many more steps) to [[https://docs.opentripplanner.org/en/latest/Basic-Tutorial/|run OpenTripPlanner standalone]] with the configuration below.
 ==== Preparation - Files ====
   - **Create a working directory for OpenTripPlanner**, for this example, we will use ''./opentripplanner'', you may wish to use different persistent storage e.g. for a Docker volume
     * ''mkdir opentripplanner && cd ./opentripplanner''
-  - **Manually download GTFS static data** (this can be automated once api.data.gov.my serves GTFS with header ''Content-Type: application/zip'')
+  - **Create configuration files**
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/mybas-johor%%''
+    - **Build Configuration** to download regional OpenStreepMap data and GTFS data (GTFS are not the documented API links [[https://github.com/opentripplanner/OpenTripPlanner/issues/6351|due to URI extension validation in OpenTripPlanner]])\\ <file json build-config.json>{
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/ktmb%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-penang%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-kuantan%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-mrtfeeder%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-rail-kl%%''
-    * ''%%wget --trust-server-names https://api.data.gov.my/gtfs-static/prasarana?category=rapid-bus-kl%%''
-  - **Create additional configuration files**
-    - Build Configuration to automatically download regional OpenStreepMap; timezone is required as Malaysia GTFS does not define timezone\\ <file json build-config.json>{
-  "transitModelTimeZone": "Asia/Kuala_Lumpur",
   "osm": [
     {
       "source": "https://download.geofabrik.de/asia/malaysia-singapore-brunei-latest.osm.pbf"
     }
-  ]
+  ],
+  "transitFeeds": [
+    {
+      "feedId": "ktmb",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/ktmb/gtfs_ktmb.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "mybas-johor",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/mybas-johor/gtfs_mybas.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-kl",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kl.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-kuantan",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_kuantan.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-mrtfeeder",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_mrtfeeder.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-bus-penang",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_bus_penang.zip",
+      "type": "gtfs"
+    },
+    {
+      "feedId": "rapid-rail-kl",
+      "source": "https://openapi-malaysia-transport.s3.ap-southeast-1.amazonaws.com/prasarana/gtfs_rapid_rail_kl.zip",
+      "type": "gtfs"
+    }
+  ],
+  "transitModelTimeZone": "Asia/Kuala_Lumpur"
 }
 </file>
@@ Line 40: / Line 140: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/mybas-johor",
       "feedId": "mybas-johor"
@@ Line 46: / Line 146: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/ktmb",
       "feedId": "ktmb"
@@ Line 52: / Line 152: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kl",
       "feedId": "rapid-bus-kl"
@@ Line 58: / Line 158: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-mrtfeeder",
       "feedId": "rapid-bus-mrtfeeder"
@@ Line 64: / Line 164: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-kuantan",
       "feedId": "rapid-bus-kuantan"
@@ Line 70: / Line 170: @@
     {
       "type": "vehicle-positions",
-      "frequencySec": 30,
+      "frequency": "PT30S",
       "url": "https://api.data.gov.my/gtfs-realtime/vehicle-position/prasarana?category=rapid-bus-penang",
       "feedId": "rapid-bus-penang"
     }
   ]
-}</file>
+}
+</file>
 ==== Preparation - Docker Compose ====
@@ Line 83: / Line 184: @@
     command: --load --serve
     environment:
-      - JAVA_TOOL_OPTIONS=-Xmx8g
+      - JAVA_TOOL_OPTIONS=-Xmx8G
     expose:
       - 8080
@@ Line 94: / Line 195: @@
 ==== Build Graph ====
-  * Docker Compose:\\ ''%%docker compose run opentripplanner --build --save%%''
+  * Docker Compose:\\ ''%%docker compose run --rm opentripplanner --build --save%%''
   * Docker:\\ ''%% docker run --rm -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --build --save%%''
@@ Line 104: / Line 205: @@
     * Docker (foreground): ''%%docker run -it --rm -p 8080:8080 -v ./opentripplanner:/var/opentripplanner docker.io/opentripplanner/opentripplanner:latest --load --serve%%''
-FIXME FIXME FIXME FIXME FIXME INCOMPLETE
+FIXME unfinished writing, a conclusion and some screenshots here would be nice
+===== Connecting to OneBusAway =====
+We will use OneBusAway as a mobile app frontend to OneTripPlanner.
+FIXME unfinished writing, OneBusAway + OpenTripPlanner works on Android, but not iOS; there was [[https://github.com/OneBusAway/otpkit|a GSoC 2024 project to support OpenTripPlanner on OneBusAway iOS]] but the app implementation never happened.

ndoo.sg

User Tools

Site Tools

Differences

Page Tools