February/March in KDE Itinerary
While not being able to travel remains a challenge for KDE Itinerary, there nevertheless has been a lot of activity since the last last summary blog, improving travel data extraction and public transport data integration among many other things.
KDE Itinerary was able to show train coach layouts in Germany for some time already, now we also have Austria covered and are progressing on Switzerland. While the data we get in Austria matched our existing model quite well, a lot more logic was needed in the UI to render what we get for Switzerland in a meaningful way as well. Additionally, train layouts are now also accessible for arrivals.
Location-based notes and warnings provided by public transport data services are now properly assigned to the right station in a journey. This also fixes handling of points of interests such as major tunnels that are sometimes injected into the journey data.
Transfer elements in KDE Itinerary’s timeline now also include the vehicle occupancy indicator. Parsing occupancy information is now also supported by more backends.
The architecture of the travel document extractor engine has seen a major rework that is about to be integrated. This addresses a number of shortcomings that are the result of this considerably outgrowing its initial needs.
- Supported documented types (text, HTML, PDF, Apple Wallet Passes, etc) and data extraction (e.g. interpreting barcodes) are now properly separated. This opens the door for e.g. also supporting barcodes embedded in HTML emails, but also for eventually splitting the generic extraction infrastructure and the travel-specific parts as discussed during Akademy 2020.
- Documents are now properly modeled as a tree, rather than just having nested extraction as an afterthought in a few places. This makes features previously only available for specific document types available generically. For example so far we would support different ticket barcodes depending on whether we found them in a PDF or via direct scanning. This also allows for better tooling as the intermediate steps are now accessible from the outside, KItinerary Workbench already received the first changes to make use of that.
Both of those changes combined will also considerably simplify exposing the complex binary train ticket formats to the extractor scripts, in particular the UIC 918.3 container format which can contain a variety of different payloads.
Another important behind the scenes change is the introduction of journey paths in KPublicTransport. That is, retrieving a (possibly very detailed) line of geographic coordinates and associated descriptions of the path of travel, for example to display that on a map.
So far this is only used to compute a more accurate travel distance, but the main motivation for implementing this is actually to get usable routing instructions for the parts of a journey you don’t spend on a train or bus: ways to and from the station/stop, and transfers. Now that we have access to the data this needs to be made accessible and properly displayed in the app.
While all virtual, there were a few events featuring KDE Itinerary or covering topics relevant for it:
- Three weeks ago I presented KDE Itinerary at the German Open Transport Meetup, the slides are available online (but might not be all that useful without the audio track).
- Shortly after there was a surprise opportunity to quickly explain what KDE Itinerary is to 100+ attendees of the DB Regio Data Hackathon. I also spotted it being mentioned there on a slide of the Open Bike Box team :)
- During last weekend’s KF6 sprint I got some feedback on the proposed API for obtaining (localized) information about countries, country subdivisions, timezones, languages and currencies, something that would help KDE Itinerary in a few places and supersede some existing code we already have for this.
Fixes & Improvements
Travel document extractor
- There is now a parser for European Railway Agency (ERA) Small Structured Barcodes (SSB), the highly compact binary format used for example in Thalys ticket barcodes. This is also exposed to JS extractor scripts, simplifying and improving the Thalys ticket extractor.
- The decoder API for VDV eTickets (and derivatives like UIC 918.3 0080VU blocks used by Deutsche Bahn City Tickets) improved to hide endian conversion from consumer code, making this much easier to use. For fully leveraging this we still lack documentation and code tables for the operator-specific coverage area blocks in there though.
- There’s a new development tool,
ticket-barcode-dump, that can print out every single detail of UIC 918.3, VDV or ERA SSB encoded binary data. That also discovered an additional complete date/time field in UIC 918.3 tickets, making ugly workarounds to recover the otherwise missing year number unnecessary. If you like digging into arcane binary formats there’s still work left here around SBB tickets, the new ERA FCB format or VDV coverage areas.
Public transport data
- We now also support the “IVV AuskunftServiceSystem” backend, which is for example used around Cologne, Germany.
- Work continued on aligning our backend service configuration format with that of the Tranport API Repository, and to make use of more information from there. In case you are interested in algorithms to compute simplified convex polygon hulls, that is the biggest remaining challenge in automatically consuming all backend services defined there.
- The OpenTripPlanner and Hafas legacy backend now also considers the current locale and their respective supported languages, resulting in more localized routing messages.
- KPublicTransport got more request parameters to fine-tune the amount and level of detail of the expected results, which can reduce unneeded network traffic.
- Adapted to changes in the weather API we use.
- Besides opening a location with the default map application there is now also the option to open that on wheelmap.org.
- The OSM opening hours parser used by the train station map supports more non-standard language variants and can recover from more mistakes in opening hours expressions.
While the pandemic makes field testing and collecting travel document samples difficult in many parts of the world, there’s plenty
of other things that can be done. The KDE Itinerary workboard or the
more specialized indoor map workboard show
what’s on the todo list, and are a good place for collecting new ideas. For questions and suggestions, please feel free
to join us on the KDE PIM mailing list or in the
#kontact channel on Matrix or Freenode.