From b2983e6d71bc214f593b23ad06483fab46be8ce1 Mon Sep 17 00:00:00 2001 From: Jeevan Topgi Date: Mon, 5 Oct 2026 19:35:45 -0400 Subject: [PATCH 1/3] Count named trail paths as hikes where trails aren't mapped as hiking routes Across the American West, trails are mostly mapped as named paths rather than hiking route relations, so searches found few hikes. The tiles query now also lists named footpaths, bridleways, and paths that aren't sidewalks, crossings, private, or part of a hiking route, with their nodes, and each set of ways of one name joined at a node is a route, with the negative of its first way's id. A picked path's details are measured from its ways, its highlights found along them, and its OpenStreetMap link goes to its first way. Paths are half a point less promising than mapped hiking routes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 29 +++- app/controllers/hikes_controller.rb | 5 +- app/services/overpass_service.rb | 199 ++++++++++++++++++++----- app/services/trails_service.rb | 15 +- app/views/hikes/_details.html.erb | 2 +- app/views/searches/_trail.html.erb | 2 +- test/integration/searches_test.rb | 20 ++- test/services/overpass_service_test.rb | 59 +++++++- test/services/route_data_test.rb | 10 ++ test/services/search_test_support.rb | 24 ++- test/services/trails_service_test.rb | 23 ++- 11 files changed, 322 insertions(+), 66 deletions(-) diff --git a/README.md b/README.md index 1fb0f4f..f87cce4 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,7 @@ local socket. Use a separate database for tests. The tables are the cache's ## External services and changed behavior The previous implementation depended on the legacy Hiking Project API. This -version instead queries hiking-route relations from +version instead queries hiking-route relations and named trail paths from [OpenStreetMap via Overpass](https://wiki.openstreetmap.org/wiki/Overpass_API). It does not scrape Hiking Project or require its old API key. Current Hiking Project availability could not be confirmed. Live probes of both @@ -116,12 +116,24 @@ test Overpass connectivity from your deployment before launching. the coarse terrain routes' promise uses, then the quickest: with buses as well as trains, flat suburbs are often as quick as the hills beyond them, and from New York they took the places of the Catskills, Poconos, and Lehigh Valley. - The four quickest are queried first, the rest four neighbors at a time, each - query finding the routes of the region around its tiles once, then keeping - those in the tiles (a tile's routes are left out when its query fails). - Routes spanning less than 300 m, under **1 mile** or over **30 miles** long, - at least half on paved paths or roads, and repeated sections of one named trail (the same - name within 5 km) are left out: they are walks or multi-day trails rather than +- **Named paths.** Where trails are mapped as named paths alone rather than as + hiking routes, as across the American West (around the Columbia River Gorge, + 30 routes against 218 named trails; in LA's San Gabriels, 8 against 135), the + tiles' named footpaths, bridleways, and paths that aren't sidewalks, + crossings, private, or part of a hiking route are routes too + (`OverpassService::PATH_FILTER`): each the ways of one name joined at a node, + listed with their nodes but not their geometry (about 3 MB for four tiles + around the Issaquah Alps or Marin). A named path's id is the negative of its + first way's, its details are measured from its ways once a search picks it, + and its highlights found along them, and its OpenStreetMap link is to that + way. A mapped hiking route is likelier to be a hike worth the trip, so a + named path is half a point less promising. +- The four quickest tiles are queried first, the rest four neighbors at a + time, each query finding the routes of the region around its tiles once, then + keeping those in the tiles (a tile's routes are left out when its query + fails). Routes spanning less than 300 m, under **1 mile** or over **30 + miles** long, at least half on paved paths or roads, and repeated sections of + one named trail (the same name within 5 km) are left out: they are walks or multi-day trails rather than day hikes. Routes within a 30-minute walk of a stop (the most the planner walks) are checked, most promising first, up to **120** per search, keeping the section of a trail that transit reaches soonest. Promise counts much as the @@ -135,7 +147,8 @@ test Overpass connectivity from your deployment before launching. and back. Without the land, an area where most routes have Wikidata entries, such as Franconia's, took most checks from Munich, and the Alps few. Results are not an exhaustive trail inventory; where few hiking - routes are mapped in OpenStreetMap near stations, there are few results. + routes or named paths are mapped in OpenStreetMap near stops, there are few + results. - Routes are checked in batches of 40, starting with the first tiles' while the others are found, and each batch's hikes show as soon as their trips there and back are known. Routes that can't be looked up at once, as when Overpass is diff --git a/app/controllers/hikes_controller.rb b/app/controllers/hikes_controller.rb index 1fbd14e..8c60067 100644 --- a/app/controllers/hikes_controller.rb +++ b/app/controllers/hikes_controller.rb @@ -7,7 +7,8 @@ class HikesController < ApplicationController # The page shows without the route's highlights, terrain, or photos when they take longer. EXTRAS_WAIT_SECONDS = 10 - # route is the OpenStreetMap relation id; from, to, and finish are + # route is the OpenStreetMap relation id, or a named path's (see + # OverpassService::PATH_FILTER), whose details its search kept; from, to, and finish are # "latitude,longitude" for the search's starting point, where transit # reaches the route, and where a hike to the far end finishes; station, if # given, is "latitude,longitude" for the station trips leave from, and @@ -23,7 +24,7 @@ def show @zone = ActiveSupport::TimeZone[params[:tz]] if params[:tz].is_a?(String) && params[:tz].length <= 64 route = Integer(params[:route], 10, exception: false) if params[:route].is_a?(String) @origin_name, @station_name = [params[:origin], params[:station_name]].map { |name| name_param(name) } - unless origin && start && route&.positive? && PLANS.include?(params[:plan]) && (finish || params[:plan] != "through") && + unless origin && start && route&.nonzero? && PLANS.include?(params[:plan]) && (finish || params[:plan] != "through") && (station || !params.key?(:station)) && leave && back_by && @zone && leave.between?(1.day.ago, 8.days.from_now) && back_by.between?(leave, leave + 1.day) return head(:bad_request) diff --git a/app/services/overpass_service.rb b/app/services/overpass_service.rb index 3b912f4..1bbfaea 100644 --- a/app/services/overpass_service.rb +++ b/app/services/overpass_service.rb @@ -11,7 +11,7 @@ module OverpassService # a walk of the stops transit reaches. Each tile's routes are shared by every search. TILE_DEGREES = 0.5 # Where tiles' routes, routes' details, and their highlights are cached, by tile or route id. - TILE_KEY = "overpass:tile:v2:".freeze + TILE_KEY = "overpass:tile:v3:".freeze ROUTE_KEY = "overpass:route:v2:".freeze HIGHLIGHTS_KEY = "overpass:highlights:v3:".freeze # At most this many tiles are searched in each band of travel time, the @@ -33,7 +33,18 @@ module OverpassService TILE_KEEP = 90.days TILE_REFRESH_AFTER = 30.days # Regions' routes and routes' geometry take a while to find, especially on the mirror. - LONG_QUERY_SECONDS = 45 + LONG_QUERY_SECONDS = 60 + # Where trails are mapped as named paths alone rather than as hiking routes, + # as across the American West, the named footpaths, bridleways, and paths + # that aren't sidewalks, crossings, or private are trails too: each the + # connected ways of one name that aren't part of a hiking route, with the + # negative of its first way's id, which no route has. + PATH_FILTER = (%(["highway"~"^(path|footway|bridleway)$"]["name"]["footway"!~"^(sidewalk|crossing)$"]) + + %(["access"!~"^(private|no)$"]["foot"!~"^(private|no)$"])).freeze + PATH_SUMMARY = "A trail mapped by OpenStreetMap contributors.".freeze + # A hiking route someone mapped is likelier to be a hike worth the trip than + # a named path, so paths are this much less promising. + PATH_PROMISE = 0.5 # Trips are checked for at most this many routes per search, in batches; Transitous # plans at most 128 destinations in one request. MAX_TRANSIT_ROUTES = 120 @@ -103,6 +114,11 @@ module OverpassService Trail = Struct.new(:name, :summary, :latitude, :longitude, :length, :osm_id, :path, :highlights, :notable, :paved, :loop, :distance, :duration, :transfers, :arrival, :last_return, :origin, :terrain, :score, :plan, :finish, :location, :station, :sunset, :noise, keyword_init: true) do + # The route's page on OpenStreetMap: its hiking route's, or a named path's first way's. + def osm_url + osm_id.negative? ? "https://www.openstreetmap.org/way/#{-osm_id}" : "https://www.openstreetmap.org/relation/#{osm_id}" + end + # A point halfway along the route, in its area even where transit reaches it from town. def midpoint points = Array(path).flatten(1) @@ -214,10 +230,12 @@ def self.tile_key(tile) "#{TILE_KEY}#{tile.join(':')}" end - # The tiles' routes, in one query, kept as { tile key => { routes:, at: } }, - # when they were looked up. + # The tiles' routes and named paths, in one query, kept as + # { tile key => { routes:, at: } }, when they were looked up. def self.fetch_tiles(tiles, connections, cache) - routes = elements(tiles_query(tiles), connections, cache, timeout: LONG_QUERY_SECONDS).filter_map { |element| candidate(element) } + ways, relations = elements(tiles_query(tiles), connections, cache, timeout: LONG_QUERY_SECONDS) + .partition { |element| element.is_a?(Hash) && element["type"] == "way" } + routes = relations.filter_map { |element| candidate(element) } + path_candidates(ways) at = Time.current tiles.to_h do |south, west| within = routes.select do |route| @@ -282,18 +300,60 @@ def self.shared(keys, cache) def self.tiles_query(tiles) region = [tiles.map(&:first).min, tiles.map(&:last).min, - tiles.map(&:first).max + TILE_DEGREES, tiles.map(&:last).max + TILE_DEGREES] - clauses = tiles.map { |south, west| "relation.region(#{south},#{west},#{south + TILE_DEGREES},#{west + TILE_DEGREES});" } - %([out:json][timeout:40];relation["type"="route"]["route"="hiking"](#{region.join(',')})->.region;) + - "(#{clauses.join});out tags bb;" + tiles.map(&:first).max + TILE_DEGREES, tiles.map(&:last).max + TILE_DEGREES].join(",") + boxes = tiles.map { |south, west| "(#{south},#{west},#{south + TILE_DEGREES},#{west + TILE_DEGREES})" } + # Named paths come with the nodes they're joined by, but not the ways of the routes, which are routes already. + %([out:json][timeout:60];relation["type"="route"]["route"="hiking"](#{region})->.region;) + + "(#{boxes.map { |box| "relation.region#{box};" }.join});out tags bb;" \ + "way(r.region)->.routed;way#{PATH_FILTER}(#{region})->.paths;" \ + "((#{boxes.map { |box| "way.paths#{box};" }.join}); - .routed;);out body bb;" + end + + # Candidates like #candidate's for the named paths among the ways, with the + # ids of their ways: those of one name joined at a node are one trail. + def self.path_candidates(ways) + ways = ways.select do |way| + way["id"].is_a?(Integer) && way["id"].positive? && way["tags"].is_a?(Hash) && name(way["tags"]) && + way["nodes"].is_a?(Array) && way["bounds"].is_a?(Hash) + end + ways.group_by { |way| name(way["tags"]).downcase }.values.flat_map do |named| + leader = named.to_h { |way| [way["id"], way["id"]] } + find = lambda do |id| + id = leader[id] while leader[id] != id + id + end + first_way = {} + named.each do |way| + way["nodes"].each do |node| + other = first_way[node] ||= way["id"] + joined = [find.(way["id"]), find.(other)] + leader[joined.max] = joined.min + end + end + named.group_by { |way| find.(way["id"]) }.values.filter_map { |trail| path_candidate(trail) } + end + end + + def self.path_candidate(ways) + boxes = ways.map { |way| way["bounds"].values_at("minlat", "minlon", "maxlat", "maxlon") } + return unless boxes.all? { |box| SearchHttp.coordinates?(*box.first(2)) && SearchHttp.coordinates?(*box.last(2)) } + + corners = [boxes.map(&:first).min, boxes.map { |box| box[1] }.min, boxes.map { |box| box[2] }.max, boxes.map(&:last).max] + span = distance(*corners) + return unless span.between?(MIN_SPAN_METERS, MAX_LENGTH_MILES * METERS_PER_MILE) + + ids = ways.map { |way| way["id"] }.sort + { id: -ids.first, name: name(ways.first["tags"]), latitude: (corners[0] + corners[2]) / 2.0, + longitude: (corners[1] + corners[3]) / 2.0, bounds: corners, span: span.round, + notable: ways.any? { |way| notable?(way["tags"]) }, ways: ids } end # The routes with these ids, measured, leaving out those too short, too long, # or mostly paved for a day hike, and joined where transit reaches them # soonest when access is known; distance is in miles from the origin. Routes - # are left out as #routes leaves them out. - def self.trails_for(ids, lat:, lon:, access: nil, connections: nil, cache: Rails.cache, failures: nil) - routes(ids, connections, cache, failures: failures).filter_map do |trail| + # are left out as #routes leaves them out, and paths are named paths' ways by id. + def self.trails_for(ids, lat:, lon:, access: nil, connections: nil, cache: Rails.cache, failures: nil, paths: {}) + routes(ids, connections, cache, failures: failures, paths: paths).filter_map do |trail| next unless trail.length.between?(MIN_LENGTH_MILES, MAX_LENGTH_MILES) && trail.paved.to_f < MOSTLY_PAVED if access @@ -355,7 +415,8 @@ def self.distinct(routes) # these, an area with many notable routes would fill every check, however far. def self.promise(route, relief = nil) hours = route[:minutes].to_f * 2 / 60 - (route[:notable] ? 1 : 0) + (route[:span] >= 800 ? 0.5 : 0) - (generic_name?(route[:name]) ? 1 : 0) + + (route[:notable] ? 1 : 0) + (route[:span] >= 800 ? 0.5 : 0) - (generic_name?(route[:name]) ? 1 : 0) - + (route[:id].negative? ? PATH_PROMISE : 0) + [relief.to_i / 100.0, 4].min * TrailsService::SCENIC_WEIGHT - TrailsService.travel_penalty(hours) end @@ -366,20 +427,21 @@ def self.generic_name?(name) # Trails for the route ids, in order. Each route's details are cached, and # only uncached routes are queried, once for searches that need them at # once. Routes that can't be looked up are left out, unless none are found, - # and their failure is added to failures when given. - def self.routes(ids, connections, cache, failures: nil) + # and their failure is added to failures when given. Named paths are + # measured from their ways, by id in paths, and left out without them. + def self.routes(ids, connections, cache, failures: nil, paths: {}) return [] if ids.empty? keys = ids.to_h { |id| [id, "#{ROUTE_KEY}#{id}"] } found = cache.read_multi(*keys.values) RouteStore.details(ids.reject { |id| found.key?(keys[id]) }).each { |id, details| found[keys[id]] = details } - missing = ids.reject { |id| found.key?(keys[id]) } + missing = ids.reject { |id| found.key?(keys[id]) || (id.negative? && !paths.key?(id)) } if missing.any? id_of = keys.invert error = nil found.merge!(shared(missing.map { |id| keys[id] }, cache) do |own| own_ids = own.map { |key| id_of[key] } - fetched = fetch_routes(own_ids, connections, cache) + fetched = fetch_routes(own_ids, connections, cache, paths.slice(*own_ids)) own_ids.to_h do |id| # Routes that cannot be measured are remembered too, so they are not queried again. cache.write(keys[id], fetched.fetch(id, false), expires_in: ROUTE_CACHE_TTL) @@ -400,26 +462,49 @@ def self.routes(ids, connections, cache, failures: nil) ids.filter_map { |id| Trail.new(**found[keys[id]].slice(*Trail.members)) if found[keys[id]] } end - # Plain route attributes by id, never transit results, using the ids of each route's paved ways. - def self.fetch_routes(ids, connections, cache) - paved = PAVED_WAYS.map { |filter| "way.ways#{filter};" } - query = "[out:json][timeout:40];relation(id:#{ids.join(',')})->.routes;.routes out geom;" \ - "way(r.routes)->.ways;(#{paved.join});out ids;" + # Plain route attributes by id, never transit results, using the ids of each + # route's paved ways: routes' from their relations, and named paths' from + # their ways, by id in paths. + def self.fetch_routes(ids, connections, cache, paths = {}) + relations, named = ids.partition(&:positive?) + path_ways = named.flat_map { |id| paths.fetch(id) }.uniq + sets = [] + query = +"[out:json][timeout:60];" + if relations.any? + query << "relation(id:#{relations.join(',')})->.routes;.routes out geom;way(r.routes)->.ways;" + sets << ".ways;" + end + if path_ways.any? + query << "way(id:#{path_ways.join(',')})->.named;.named out tags geom;" + sets << ".named;" + end + return {} if sets.empty? + + query << "(#{sets.join})->.measured;(#{PAVED_WAYS.map { |filter| "way.measured#{filter};" }.join});out ids;" elements = elements(query, connections, cache, timeout: LONG_QUERY_SECONDS) - .group_by { |element| element["type"] if element.is_a?(Hash) } - paved = elements.fetch("way", []).pluck("id").to_set - elements.except("way").values.flatten(1).each_with_object({}) do |element, routes| + ways, others = elements.partition { |element| element.is_a?(Hash) && element["type"] == "way" } + # Paved ways are listed by id alone, after the named paths' ways and their geometry. + with_geometry, paved = ways.partition { |way| way.key?("geometry") } + paved = paved.pluck("id").to_set + routes = others.each_with_object({}) do |element, found| route = route_attributes(element, paved) - routes[route[:osm_id]] = route if route + found[route[:osm_id]] = route if route + end + by_id = with_geometry.index_by { |way| way["id"] } + named.each do |id| + path = path_attributes(id, paths.fetch(id).filter_map { |way| by_id[way] }, paved) + routes[id] = path if path end + routes end # Highlights near each trail, as { osm_id => [{ kind:, name:, notable:, height: }] } # without the attributes they don't have: waterfalls first, then famous and # named ones before the others. notable is true for highlights with a Wikipedia # article, and height is a waterfall's in meters. Each route's list is cached, - # and only uncached routes are queried. Routes whose highlights can't be - # looked up are left out, unless none are found. + # and only uncached routes are queried, named paths along the ways their + # details list. Routes whose highlights can't be looked up are left out, + # unless none are found. def self.highlights(trails, connections: nil, cache: Rails.cache) return {} if trails.empty? @@ -427,9 +512,16 @@ def self.highlights(trails, connections: nil, cache: Rails.cache) found = cache.read_multi(*keys.values) RouteStore.highlights(trails.map(&:osm_id).reject { |id| found.key?(keys[id]) }).each { |id, list| found[keys[id]] = list } missing = trails.reject { |trail| found.key?(keys[trail.osm_id]) } + path_ways = self.path_ways(missing.map(&:osm_id).select(&:negative?), cache) + missing.select! { |trail| trail.osm_id.positive? || path_ways.key?(trail.osm_id) } if missing.any? filters = HIGHLIGHT_TAGS.values.map { |key, value| %(node(around.ways:#{HIGHLIGHT_METERS})["#{key}"="#{value}"];) } - query = "[out:json][timeout:20];relation(id:#{missing.map(&:osm_id).join(',')});way(r)->.ways;(#{filters.join});out;" + relations = missing.map(&:osm_id).select(&:positive?) + sets = {} + sets[".routed"] = "relation(id:#{relations.join(',')});way(r)->.routed;" if relations.any? + sets[".named"] = "way(id:#{path_ways.values.flatten.uniq.join(',')})->.named;" if path_ways.any? + query = "[out:json][timeout:20];#{sets.values.join}(#{sets.keys.map { |set| "#{set};" }.join})->.ways;" \ + "(#{filters.join});out;" connections ||= [SearchHttp.connection(urls(cache).first, timeout: HIGHLIGHT_TIMEOUT_SECONDS)] wait = ProviderSlots.priority == ProviderSlots::BACKGROUND ? SLOT_WAIT_SECONDS : 0 begin @@ -445,6 +537,16 @@ def self.highlights(trails, connections: nil, cache: Rails.cache) keys.transform_values { |key| found[key] }.compact end + # The ways of the named paths with these ids, as their details list them, by id. + def self.path_ways(ids, cache) + return {} if ids.empty? + + keys = ids.to_h { |id| [id, "#{ROUTE_KEY}#{id}"] } + found = cache.read_multi(*keys.values) + stored = RouteStore.details(ids.reject { |id| found[keys[id]] }) + ids.to_h { |id| [id, Array((found[keys[id]] || stored[id] || {})[:ways])] }.reject { |_, ways| ways.empty? } + end + def self.route_attributes(element, paved) tags = hiking_tags(element) members = element["members"] @@ -459,16 +561,13 @@ def self.route_attributes(element, paved) ways = ways.uniq { |way| way["ref"] } return unless ways.all? { |way| valid_geometry?(way["geometry"]) } - lengths = ways.map do |way| - way["geometry"].each_cons(2).sum { |first, last| distance(first["lat"], first["lon"], last["lat"], last["lon"]) } - end + lengths = ways.map { |way| way_length(way) } meters = lengths.sum return unless meters.positive? first_way = ways.first start = first_way["role"] == "backward" ? first_way["geometry"].last : first_way["geometry"].first - # Every way end meets another in a loop, while a line has two loose ends. - ends = ways.flat_map { |way| way["geometry"].values_at(0, -1) }.map { |point| [point["lat"], point["lon"]] }.tally + ends = way_ends(ways) { name: name(tags) || UNNAMED, summary: tags["description"].is_a?(String) ? tags["description"] : "A hiking route mapped by OpenStreetMap contributors.", @@ -478,6 +577,38 @@ def self.route_attributes(element, paved) } end + # A named path's attributes for a Trail, like #route_attributes', from its + # ways with their geometry and tags, with the ids of its ways, which its + # highlights are found along. A line starts at one of its loose ends. + def self.path_attributes(id, ways, paved) + ways = ways.select { |way| way["id"].is_a?(Integer) && way["tags"].is_a?(Hash) && valid_geometry?(way["geometry"]) } + lengths = ways.map { |way| way_length(way) } + meters = lengths.sum + return unless meters.positive? + + ends = way_ends(ways) + start = ends.find { |_, count| count.odd? }&.first || ends.keys.first + tags = ways.map { |way| way["tags"] } + description = tags.map { |each| each["description"] }.find { |text| text.is_a?(String) && text.strip.present? } + { + name: tags.filter_map { |each| name(each) }.first || UNNAMED, summary: description || PATH_SUMMARY, + latitude: start[0], longitude: start[1], length: meters / METERS_PER_MILE, osm_id: id, + path: preview_path(ways), notable: tags.any? { |each| notable?(each) }, loop: ends.values.all?(&:even?), + paved: (ways.zip(lengths).sum { |way, length| paved.include?(way["id"]) ? length : 0 } / meters).round(2), + ways: ways.map { |way| way["id"] } + } + end + + def self.way_length(way) + way["geometry"].each_cons(2).sum { |first, last| distance(first["lat"], first["lon"], last["lat"], last["lon"]) } + end + + # How many of the ways end at each point: every way end meets another in a + # loop, while a line has two loose ends. + def self.way_ends(ways) + ways.flat_map { |way| way["geometry"].values_at(0, -1) }.map { |point| [point["lat"], point["lon"]] }.tally + end + # A hiking route relation's tags, or nil for other relations. def self.hiking_tags(element) unless element.is_a?(Hash) && element["type"] == "relation" && element["id"].is_a?(Integer) && diff --git a/app/services/trails_service.rb b/app/services/trails_service.rb index ac41aef..7b70581 100644 --- a/app/services/trails_service.rb +++ b/app/services/trails_service.rb @@ -227,12 +227,14 @@ def self.from_station(station, departure_time, transit: TransitousService, hikin noise = noise_at(station, noise) search = Search.new(station, place, access, result, transit, hiking, elevation, noise, on_found) routes = finished(nearby).value! + search.known(routes) relief = reliefs(routes, elevation) search.check(hiking.pick(routes, access: access, relief: relief).first(BATCH_SIZE)) if farther more = optional { finished(farther).value! } result.complete = false unless more more = Array(more).reject { |route| relief.key?(route[:id]) } + search.known(more) relief = relief.merge(reliefs(more, elevation)) routes = (routes + more).uniq { |route| route[:id] } end @@ -255,10 +257,17 @@ def initialize(station, place, access, result, transit, hiking, elevation, noise @station, @place, @access, @result, @transit, @hiking, @on_found = station, place, access, result, transit, hiking, on_found @elevation, @noise = elevation, noise @checked, @lookups, @budget = Set.new, [], { planned: MAX_PLANNED_ROUTES } + # The ways of the named paths among the routes, which their details are measured from. + @paths = {} # Once planning trips is stuck, the search plans no more. @stuck = Concurrent::AtomicBoolean.new end + # Notes the ways of the named paths among the routes the search may check. + def known(routes) + routes.each { |route| @paths[route[:id]] = route[:ways] if route[:ways] } + end + def check(ids) ids = ids.reject { |id| @checked.include?(id) }.first(OverpassService::MAX_TRANSIT_ROUTES - @checked.size) ids.each_slice(BATCH_SIZE) { |batch| check_batch(batch) } @@ -301,7 +310,8 @@ def plan(trails) # without asking Overpass. Routes that still can't be are left out. def routes(ids) failures = [] - found = @hiking.trails_for(ids, lat: @place.latitude, lon: @place.longitude, access: @access, failures: failures) + found = @hiking.trails_for(ids, lat: @place.latitude, lon: @place.longitude, access: @access, failures: failures, + paths: @paths.slice(*ids)) return found if failures.empty? sleep Rails.configuration.x.overpass_retry_pause_seconds @@ -317,7 +327,8 @@ def routes(ids) # leaving the search incomplete without the rest. def again(ids) failures = [] - found = @hiking.trails_for(ids, lat: @place.latitude, lon: @place.longitude, access: @access, failures: failures) + found = @hiking.trails_for(ids, lat: @place.latitude, lon: @place.longitude, access: @access, failures: failures, + paths: @paths.slice(*ids)) fail_with(failures.first) if failures.any? found rescue SearchErrors::UpstreamError => failure diff --git a/app/views/hikes/_details.html.erb b/app/views/hikes/_details.html.erb index 194b771..deae5de 100644 --- a/app/views/hikes/_details.html.erb +++ b/app/views/hikes/_details.html.erb @@ -77,7 +77,7 @@ <%= link_to "🧭 Directions back", directions_back_url(trail, place), class: "btn btn-success btn-sm", target: "_blank", rel: "noopener" %> <% end %> - <%= link_to "View on OpenStreetMap", "https://www.openstreetmap.org/relation/#{trail.osm_id}", + <%= link_to "View on OpenStreetMap", trail.osm_url, class: "btn btn-outline-secondary btn-sm", target: "_blank", rel: "noopener" %> diff --git a/app/views/searches/_trail.html.erb b/app/views/searches/_trail.html.erb index 11e38f4..6c1676e 100644 --- a/app/views/searches/_trail.html.erb +++ b/app/views/searches/_trail.html.erb @@ -78,7 +78,7 @@ <%# Shorter on phones, so the buttons fit on a line. %> <%= link_to hike_details_path(trail, result), class: "btn btn-outline-success btn-sm", aria: { label: "Details and timetables for #{trail.name}" } do %>Details & timetables<% end %> - <%= link_to "https://www.openstreetmap.org/relation/#{trail.osm_id}", class: "btn btn-outline-secondary btn-sm", + <%= link_to trail.osm_url, class: "btn btn-outline-secondary btn-sm", target: "_blank", rel: "noopener", aria: { label: "#{trail.name} on OpenStreetMap" } do %>View on OpenStreetMap<% end %> diff --git a/test/integration/searches_test.rb b/test/integration/searches_test.rb index c6983a4..73b4f80 100644 --- a/test/integration/searches_test.rb +++ b/test/integration/searches_test.rb @@ -679,6 +679,24 @@ def hike_params(**changes) assert_select ".trail-location", count: 0 end + test "a named path's page has its highlights from along its ways, and links to its first way on OpenStreetMap" do + HikingRoute.create!(osm_id: -5, collected_at: Time.current, details: { osm_id: -5, name: "Mount Si Trail", + summary: OverpassService::PATH_SUMMARY, latitude: 47.3, longitude: -122.0, length: 1.38, path: [[[47.3, -122.0], [47.32, -122.0]]], + loop: false, paved: 0.0, notable: false, ways: [5] }) + hiking([], highlights: [highlight_node("waterfall", 47.31, -122.0, name: "Twin Falls")]) + elevation + stub_get(WikipediaService::API_URL, {}) + stub_get(PhotonService::REVERSE_URL, { features: [] }) + planner(window: [train_there("15:19", "17:31")], back: [train_back("20:00", "21:30")]) + get hike_path(hike_params(route: -5)) + + assert_response :success + assert_select "h1", text: "Mount Si Trail" + assert_select ".trail-chip", text: /Twin Falls/ + assert_select "a[href='https://www.openstreetmap.org/way/5']", text: /OpenStreetMap/ + assert(@requests["overpass"].any? { |query| query.include?("way(id:5)->.named;") }) + end + test "a hike's page shows its route, the trips there from its station that leave time to hike all of it, and the trips back" do hiking([route_element(latitude: 47.3, name: "Ridge Trail")], highlights: [highlight_node("waterfall", 47.31, -122.0, name: "Twin Falls")]) elevation @@ -749,7 +767,7 @@ def hike_params(**changes) end test "a hike's page needs a route, points, a plan, times on one day, and a time zone, and says when it can't plan" do - [{ route: "abc" }, { plan: "wander" }, { plan: "through" }, { from: "91,0" }, { to: nil }, { tz: "Mars/Olympus" }, + [{ route: "abc" }, { route: "0" }, { plan: "wander" }, { plan: "through" }, { from: "91,0" }, { to: nil }, { tz: "Mars/Olympus" }, { leave: "soon" }, { back_by: "2026-09-26T06:00:00Z" }, { station: "north" }].each do |change| get hike_path(hike_params(**change).compact) assert_response :bad_request diff --git a/test/services/overpass_service_test.rb b/test/services/overpass_service_test.rb index 20d0ae1..09814f7 100644 --- a/test/services/overpass_service_test.rb +++ b/test/services/overpass_service_test.rb @@ -127,9 +127,13 @@ def centered(row, column, minutes) found = routes_in(connection, tiles: [[47.0, -122.0], [47.5, -122.0], [48.5, -122.5]], cache: cache) assert_equal [1, 2], found.pluck(:id).sort - assert_equal '[out:json][timeout:40];relation["type"="route"]["route"="hiking"](47.0,-122.5,49.0,-121.5)->.region;' \ - "(relation.region(47.0,-122.0,47.5,-121.5);relation.region(47.5,-122.0,48.0,-121.5);" \ - "relation.region(48.5,-122.5,49.0,-122.0););out tags bb;", queries.sole + region = "(47.0,-122.5,49.0,-121.5)" + boxes = ["(47.0,-122.0,47.5,-121.5)", "(47.5,-122.0,48.0,-121.5)", "(48.5,-122.5,49.0,-122.0)"] + # Named paths come too, but not the ways of hiking routes. + assert_equal %([out:json][timeout:60];relation["type"="route"]["route"="hiking"]#{region}->.region;) + + "(#{boxes.map { |box| "relation.region#{box};" }.join});out tags bb;" \ + "way(r.region)->.routed;way#{OverpassService::PATH_FILTER}#{region}->.paths;" \ + "((#{boxes.map { |box| "way.paths#{box};" }.join}); - .routed;);out body bb;", queries.sole # The route across two tiles is in both. assert_equal [2], routes_in(connection, tiles: [[47.5, -122.0]], cache: cache).pluck(:id) assert_equal [1, 2], routes_in(connection, tiles: [[47.5, -122.0], [47.0, -122.0]], cache: cache).pluck(:id).sort @@ -141,7 +145,7 @@ def centered(row, column, minutes) # A month on, the kept routes are used at once while the tile is looked up again in the background. travel 30.days + 1.minute assert_equal [2], routes_in(connection, tiles: [[47.5, -122.0]], cache: cache).pluck(:id) - Timeout.timeout(5) { sleep 0.01 until cache.read("overpass:tile:v2:47.5:-122.0")[:at] > 1.minute.ago } + Timeout.timeout(5) { sleep 0.01 until cache.read("#{OverpassService::TILE_KEY}47.5:-122.0")[:at] > 1.minute.ago } assert_equal 3, queries.size assert_includes queries.last, "(47.5,-122.0,48.0,-121.5)->.region;" routes_in(connection, tiles: [[47.5, -122.0]], cache: cache) @@ -176,7 +180,7 @@ def centered(row, column, minutes) racing = stub_connection(:post, lambda { |request| query = URI.decode_www_form(request.body).to_h.fetch("data") queries << query - cache.write("overpass:tile:v2:51.0:-122.0", { routes: [{ id: 7 }], at: Time.current }) if query.include?("relation.region(49.0,") + cache.write("#{OverpassService::TILE_KEY}51.0:-122.0", { routes: [{ id: 7 }], at: Time.current }) if query.include?("relation.region(49.0,") { "elements" => [] } }) assert_equal [7], routes_in(racing, tiles: later, cache: cache).pluck(:id) @@ -248,7 +252,7 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events reads.reset waiting = Thread.new { routes_in(connection, tiles: [[47.0, -122.0]], cache: cache) } assert reads.wait(5) - cache.write("overpass:tile:v2:47.0:-122.0", { routes: [{ id: 9 }], at: Time.current }) + cache.write("#{OverpassService::TILE_KEY}47.0:-122.0", { routes: [{ id: 9 }], at: Time.current }) assert_equal [9], waiting.join(5).value.pluck(:id) end # Only the first search asked Overpass. @@ -450,6 +454,45 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events assert_empty OverpassService.pick(near + far, access: access, checked: (1..120).to_set) end + test "named paths that aren't hiking routes are routes too, each the ways of one name joined at a node" do + paths = [path_way(1, nodes: [10, 11]), path_way(2, nodes: [11, 12], latitude: 47.02), + # The same name elsewhere is another trail, and a short path isn't one. + path_way(3, nodes: [30, 31], latitude: 47.3), path_way(4, name: "Lot Path", nodes: [40, 41], latitude: 47.4, length: 0.001)] + found = routes_in(overpass_connection(routes: [route_element(id: 9)], paths: paths), cache: ActiveSupport::Cache::MemoryStore.new) + assert_equal [[9, nil], [-1, [1, 2]], [-3, [3]]], found.map { |route| route.values_at(:id, :ways) } + joined = found.find { |route| route[:id] == -1 } + assert_equal ["Mount Si Trail", [47.0, -122.0, 47.04, -122.0], 4448, false], + [joined[:name], joined[:bounds].map { |degrees| degrees.round(6) }, joined[:span], joined[:notable]] + end + + test "a named path is measured from its ways and kept with them, and its highlights are found along them" do + cache = ActiveSupport::Cache::MemoryStore.new + queries = [] + paths = [path_way(1, nodes: [10, 11], length: 0.03), path_way(2, nodes: [11, 12], latitude: 47.03, surface: "asphalt")] + peak = highlight_node("peak", 47.05, -122.0, name: "Mount Si") + connection = overpass_connection(routes: [], paths: paths, paved: [2], highlights: [peak], queries: queries) + trail = OverpassService.trails_for([-1], lat: 47.0, lon: -122.0, connections: [connection], cache: cache, + paths: { -1 => [1, 2] }).sole + assert_includes queries.last, "way(id:1,2)->.named;.named out tags geom;(.named;)->.measured;" + assert_equal ["Mount Si Trail", false, 0.4, "https://www.openstreetmap.org/way/1"], [trail.name, trail.loop, trail.paved, trail.osm_url] + assert_in_delta 3.45, trail.length, 0.01 + # It starts at a loose end. + assert_includes [47.0, 47.05], trail.latitude + assert_equal [1, 2], cache.read("#{OverpassService::ROUTE_KEY}-1")[:ways] + + assert_equal({ -1 => [{ kind: "peak", name: "Mount Si" }] }, OverpassService.highlights([trail], connections: [connection], cache: cache)) + assert_includes queries.last, "way(id:1,2)->.named;(.named;)->.ways;" + + # A named path whose ways aren't known isn't looked up, or written off as unmeasurable. + assert_empty OverpassService.trails_for([-5], lat: 47.0, lon: -122.0, connections: [connection], cache: cache) + assert_equal 2, queries.size + assert_not cache.exist?("#{OverpassService::ROUTE_KEY}-5") + end + + test "a named path is a little less promising than a hiking route alike" do + assert_equal [1, -1], pick([candidate(-1, 10, name: "Cougar Ridge"), candidate(1, 10, name: "Squak Ridge")]) + end + test "promising routes are checked first" do routes = [candidate(1, 2, name: "Trail 1"), candidate(2, 3, span: 500), candidate(3, 4, notable: true, span: 500), candidate(4, 5)] @@ -489,7 +532,7 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events trail = fetch([route], paved: [456], queries: queries).sole assert_includes queries.last, "relation(id:123)->.routes;.routes out geom;way(r.routes)->.ways;" - assert_includes queries.last, 'way.ways["footway"="sidewalk"];' + assert_includes queries.last, 'way.measured["footway"="sidewalk"];' assert_equal "Forest Loop", trail.name assert_equal "A wooded walk", trail.summary assert_equal [47.0, -122.0], [trail.latitude, trail.longitude] @@ -624,7 +667,7 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events connection = overpass_connection(highlights: nodes, queries: queries) highlights = OverpassService.highlights(trails, connections: [connection], cache: Rails.cache) - assert_includes queries.sole, "relation(id:1,2);way(r)->.ways;" + assert_includes queries.sole, "relation(id:1,2);way(r)->.routed;(.routed;)->.ways;" assert_includes queries.sole, 'node(around.ways:150)["waterway"="waterfall"];' assert_equal({ 1 => [{ kind: "waterfall" }, { kind: "peak", name: "Knob" }, { kind: "viewpoint", name: "Lookout" }, { kind: "viewpoint" }], 2 => [] }, highlights) diff --git a/test/services/route_data_test.rb b/test/services/route_data_test.rb index 07a066a..1954117 100644 --- a/test/services/route_data_test.rb +++ b/test/services/route_data_test.rb @@ -105,6 +105,16 @@ def import(recorder, at: Time.utc(2026, 10, 7, 10)) HikingRoute.singleton_class.remove_method(:where) if HikingRoute.singleton_class.method_defined?(:where, false) end + test "named paths are collected and stored like routes, with their ways" do + recorder = RouteData::Recorder.new + recorder.write("#{OverpassService::ROUTE_KEY}-5", { osm_id: -5, name: "Mount Si Trail", summary: OverpassService::PATH_SUMMARY, + latitude: 47.3, longitude: -122.0, length: 3.4, path: PATH, loop: false, paved: 0.0, notable: false, ways: [5, 6] }) + assert_equal 1, import(recorder) + assert_equal "Mount Si Trail", OverpassService.trails_for([-5], lat: 47.0, lon: -122.0, connections: [unreachable], + cache: ActiveSupport::Cache::MemoryStore.new).sole.name + assert_equal({ -5 => [5, 6] }, OverpassService.path_ways([-5], ActiveSupport::Cache::MemoryStore.new)) + end + test "a newer file replaces what was stored of each route, and keeps the rest" do import(collected) newer = RouteData::Recorder.new diff --git a/test/services/search_test_support.rb b/test/services/search_test_support.rb index 4d274db..33f2bae 100644 --- a/test/services/search_test_support.rb +++ b/test/services/search_test_support.rb @@ -33,16 +33,32 @@ def highlight_node(kind, latitude, longitude, name: nil, id: rand(1..1_000_000)) # The tiles, route details, or highlights response for an Overpass query, # built from full route elements. - def overpass_elements(query, routes:, highlights: [], paved: []) + # Named paths' ways are listed with the tiles, and with their geometry when asked for by id. + def overpass_elements(query, routes:, highlights: [], paved: [], paths: []) if query.include?("out tags bb") - routes.map { |route| candidate_of(route) } - elsif query.include?("out geom") - routes + paved.map { |id| { "type" => "way", "id" => id } } + routes.map { |route| candidate_of(route) } + paths.map { |way| listed_way(way) } + elsif query.include?("out ids") + named = query[/way\(id:([\d,]+)\)->\.named/, 1].to_s.split(",").map(&:to_i) + (query.include?("relation(id:") ? routes : []) + paths.select { |way| named.include?(way["id"]) } + + paved.map { |id| { "type" => "way", "id" => id } } else highlights end end + # A named path's way, with its nodes and geometry, north from latitude. + def path_way(id, name: "Mount Si Trail", nodes: [id * 10, id * 10 + 1], latitude: 47.0, length: 0.02, **tags) + { "type" => "way", "id" => id, "tags" => { "highway" => "path", "name" => name, **tags }, "nodes" => nodes, + "geometry" => [{ "lat" => latitude, "lon" => -122.0 }, { "lat" => latitude + length, "lon" => -122.0 }] } + end + + # A way as the tiles query lists it: with its nodes and bounds, but not its geometry. + def listed_way(way) + latitudes, longitudes = way["geometry"].pluck("lat"), way["geometry"].pluck("lon") + way.except("geometry").merge("bounds" => { "minlat" => latitudes.min, "minlon" => longitudes.min, + "maxlat" => latitudes.max, "maxlon" => longitudes.max }) + end + # A one-to-all entry: a stop served by modes, reached after minutes and rides. def reached_stop(latitude, longitude, minutes, rides: 1, modes: ["REGIONAL_RAIL"], id: nil, importance: nil) { "place" => { "lat" => latitude, "lon" => longitude, "modes" => modes, "stopId" => id, "importance" => importance }.compact, diff --git a/test/services/trails_service_test.rb b/test/services/trails_service_test.rb index 1aeccb7..d32decf 100644 --- a/test/services/trails_service_test.rb +++ b/test/services/trails_service_test.rb @@ -138,7 +138,7 @@ def named(place) class FakeHiking FIRST_TILE = [47.5, -122.5].freeze - attr_reader :access, :batches, :checked_before, :reliefs, :stations, :tile_relief, :tile_requests, :threads + attr_reader :access, :batches, :checked_before, :paths_given, :reliefs, :stations, :tile_relief, :tile_requests, :threads # stuck names the lookups that wait for release: :first_tiles, :other_tiles, :trails_for, or :highlights. # A lookup of routes including a failing one fails, and unstored routes are left out of a lookup that finds @@ -148,7 +148,7 @@ def initialize(trails, far: [], highlights: {}, failing: [], unstored: [], busy: stuck: [:highlights], tiles_failing: [], crowded: false) @trails, @far, @highlights, @failing, @release, @stuck = trails, far, highlights, failing, release, stuck @unstored, @busy, @tiles_failing, @crowded = unstored, busy, tiles_failing, crowded - @batches, @threads, @tile_requests, @reliefs, @checked_before = [], [], Concurrent::Array.new, [], [] + @batches, @threads, @tile_requests, @reliefs, @checked_before, @paths_given = [], [], Concurrent::Array.new, [], [], [] end # One tile, or five when there are routes in the others; crowded ones are picked knowing their relief. @@ -168,7 +168,11 @@ def routes_in(tiles, failures: nil) failures&.push(SearchErrors::UpstreamError.new("busy")) if @tiles_failing.include?(first ? :first_tiles : :other_tiles) - (first ? @trails : @far).map { |trail| { id: trail.osm_id, latitude: trail.latitude, longitude: trail.longitude } } + # Named paths, with negative ids, list their ways. + (first ? @trails : @far).map do |trail| + { id: trail.osm_id, latitude: trail.latitude, longitude: trail.longitude, ways: ([-trail.osm_id] if trail.osm_id.negative?) } + .compact + end end def pick(routes, access:, relief:, checked: Set.new) @@ -178,10 +182,11 @@ def pick(routes, access:, relief:, checked: Set.new) routes.pluck(:id) end - def trails_for(ids, lat:, lon:, access:, failures: nil) + def trails_for(ids, lat:, lon:, access:, failures: nil, paths: {}) @release&.wait(5) if @stuck.include?(:trails_for) @threads << Thread.current @batches << ids + @paths_given << paths raise SearchErrors::UpstreamError, "busy" if ids.intersect?(@failing) found = @batches.count { |batch| batch.intersect?(@unstored) } > @busy ? ids : ids - @unstored @@ -261,7 +266,7 @@ def trail(name, length: 3.0, distance: 30.0, loop: true, at: nil, **attributes) @trails_made = @trails_made.to_i + 1 latitude = at || 47.0 + @trails_made * 0.1 OverpassService::Trail.new(name: name, latitude: latitude, longitude: -122.1, length: length, distance: distance, - osm_id: name.hash, path: [[[latitude, -122.1], [latitude + 0.01, -122.1]]], loop: loop, paved: 0, **attributes) + osm_id: name.hash.abs, path: [[[latitude, -122.1], [latitude + 0.01, -122.1]]], loop: loop, paved: 0, **attributes) end def search(origin: "Seattle", day: nil, near: nil, places: FakePlaces.new, transit: FakeTransit.new, @@ -318,6 +323,14 @@ def minutes(count) assert_equal [{}, {}], hiking.reliefs end + test "named paths' details are looked up from the ways their tiles list" do + route, path = trail("route"), trail("path", osm_id: -77) + hiking = FakeHiking.new([route, path]) + result = search(transit: FakeTransit.new(trips: { "route" => minutes(30), "path" => minutes(30) }), hiking: hiking) + assert_equal %w[path route], result.trails.map(&:name).sort + assert_equal [{ -77 => [77] }], hiking.paths_given + end + test "tiles in bands of travel time with more than their share are picked knowing how far the land rises across them" do hiking = FakeHiking.new([trail("a")], crowded: true) elevation = FakeElevation.new(relief: { [47.5, -122.5] => 300 }) From d59d9710b0343a3f3803c46e9877a57b269b2a40 Mon Sep 17 00:00:00 2001 From: Jeevan Topgi Date: Mon, 5 Oct 2026 20:07:14 -0400 Subject: [PATCH 2/3] Keep tiles' routes where named paths are too many, and find trails across tile edges whole MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Where named paths would take a tile query over 8 MB, as around Zürich, the tiles are queried again for their hiking routes alone, and a response that large isn't asked of the other instance. Paths are listed 0.05° beyond their tiles, pieces of one trail from different tiles are joined where they share a way, and a path measured from fewer of its ways than a search knows is measured again from them all. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 16 +++-- app/services/overpass_service.rb | 98 +++++++++++++++++++------- app/services/trails_service.rb | 5 +- config/guides.yml | 6 +- test/services/overpass_service_test.rb | 69 +++++++++++++++--- 5 files changed, 148 insertions(+), 46 deletions(-) diff --git a/README.md b/README.md index f87cce4..de48911 100644 --- a/README.md +++ b/README.md @@ -123,11 +123,17 @@ test Overpass connectivity from your deployment before launching. crossings, private, or part of a hiking route are routes too (`OverpassService::PATH_FILTER`): each the ways of one name joined at a node, listed with their nodes but not their geometry (about 3 MB for four tiles - around the Issaquah Alps or Marin). A named path's id is the negative of its - first way's, its details are measured from its ways once a search picks it, - and its highlights found along them, and its OpenStreetMap link is to that - way. A mapped hiking route is likelier to be a hike worth the trip, so a - named path is half a point less promising. + around the Issaquah Alps or Marin), and 0.05° beyond the tiles, so a trail + across a tile's edge is found whole from either side. Pieces of one that + tiles list apart are joined where they share a way, and a path measured from + fewer of its ways than a search knows is measured again from them all. A + named path's id is the negative of its first way's, its details are measured + from its ways once a search picks it, and its highlights found along them, + and its OpenStreetMap link is to that way. A mapped hiking route is likelier + to be a hike worth the trip, so a named path is half a point less promising. + Where named paths are too many to list with the routes (over 8 MB), as around + Zürich, which has hiking routes aplenty, tiles keep their routes alone, and a + response that large isn't asked of the other instance. - The four quickest tiles are queried first, the rest four neighbors at a time, each query finding the routes of the region around its tiles once, then keeping those in the tiles (a tile's routes are left out when its query diff --git a/app/services/overpass_service.rb b/app/services/overpass_service.rb index 1bbfaea..c3336e4 100644 --- a/app/services/overpass_service.rb +++ b/app/services/overpass_service.rb @@ -42,6 +42,9 @@ module OverpassService PATH_FILTER = (%(["highway"~"^(path|footway|bridleway)$"]["name"]["footway"!~"^(sidewalk|crossing)$"]) + %(["access"!~"^(private|no)$"]["foot"!~"^(private|no)$"])).freeze PATH_SUMMARY = "A trail mapped by OpenStreetMap contributors.".freeze + # Paths are listed this far beyond their tiles too, so a trail across a + # tile's edge is found whole, and the same, from either side. + PATH_MARGIN_DEGREES = 0.05 # A hiking route someone mapped is likelier to be a hike worth the trip than # a named path, so paths are this much less promising. PATH_PROMISE = 0.5 @@ -223,7 +226,8 @@ def self.routes_in(tiles, connections: nil, cache: Rails.cache, failures: nil) end raise error if error && keys.values.none? { |key| found.key?(key) } - keys.values.filter_map { |key| found[key]&.dig(:routes) }.flatten(1).uniq { |route| route[:id] } + paths, routes = keys.values.filter_map { |key| found[key]&.dig(:routes) }.flatten(1).partition { |route| route[:ways] } + routes.uniq { |route| route[:id] } + joined_paths(paths) end def self.tile_key(tile) @@ -231,10 +235,16 @@ def self.tile_key(tile) end # The tiles' routes and named paths, in one query, kept as - # { tile key => { routes:, at: } }, when they were looked up. + # { tile key => { routes:, at: } }, when they were looked up. Where named + # paths are too many to list with the routes, as around Zürich, which has + # hiking routes aplenty, the tiles keep their routes alone. def self.fetch_tiles(tiles, connections, cache) - ways, relations = elements(tiles_query(tiles), connections, cache, timeout: LONG_QUERY_SECONDS) - .partition { |element| element.is_a?(Hash) && element["type"] == "way" } + found = begin + elements(tiles_query(tiles), connections, cache, timeout: LONG_QUERY_SECONDS) + rescue SearchErrors::ResponseTooLarge + elements(tiles_query(tiles, paths: false), connections, cache, timeout: LONG_QUERY_SECONDS) + end + ways, relations = found.partition { |element| element.is_a?(Hash) && element["type"] == "way" } routes = relations.filter_map { |element| candidate(element) } + path_candidates(ways) at = Time.current tiles.to_h do |south, west| @@ -298,15 +308,21 @@ def self.shared(keys, cache) late.any? ? found.merge(cache.read_multi(*late)) : found end - def self.tiles_query(tiles) - region = [tiles.map(&:first).min, tiles.map(&:last).min, - tiles.map(&:first).max + TILE_DEGREES, tiles.map(&:last).max + TILE_DEGREES].join(",") - boxes = tiles.map { |south, west| "(#{south},#{west},#{south + TILE_DEGREES},#{west + TILE_DEGREES})" } - # Named paths come with the nodes they're joined by, but not the ways of the routes, which are routes already. - %([out:json][timeout:60];relation["type"="route"]["route"="hiking"](#{region})->.region;) + - "(#{boxes.map { |box| "relation.region#{box};" }.join});out tags bb;" \ - "way(r.region)->.routed;way#{PATH_FILTER}(#{region})->.paths;" \ - "((#{boxes.map { |box| "way.paths#{box};" }.join}); - .routed;);out body bb;" + def self.tiles_query(tiles, paths: true) + # A [south, west, north, east] box, margin degrees wider all around. + box = lambda do |south, west, north, east, margin = 0| + [south - margin, west - margin, north + margin, east + margin].map { |degrees| degrees.round(6) }.join(",") + end + region = [tiles.map(&:first).min, tiles.map(&:last).min, tiles.map(&:first).max + TILE_DEGREES, tiles.map(&:last).max + TILE_DEGREES] + corners = tiles.map { |south, west| [south, west, south + TILE_DEGREES, west + TILE_DEGREES] } + query = %([out:json][timeout:60];relation["type"="route"]["route"="hiking"](#{box.(*region)})->.region;) + + "(#{corners.map { |corner| "relation.region(#{box.(*corner)});" }.join});out tags bb;" + return query unless paths + + # Named paths come with the nodes they're joined by, but not the ways of hiking routes, which are routes already. + wider = box.(*region, PATH_MARGIN_DEGREES) + query + %(relation["type"="route"]["route"="hiking"](#{wider});way(r)->.routed;way#{PATH_FILTER}(#{wider})->.paths;) + + "((#{corners.map { |corner| "way.paths(#{box.(*corner, PATH_MARGIN_DEGREES)});" }.join}); - .routed;);out body bb;" end # Candidates like #candidate's for the named paths among the ways, with the @@ -317,21 +333,40 @@ def self.path_candidates(ways) way["nodes"].is_a?(Array) && way["bounds"].is_a?(Hash) end ways.group_by { |way| name(way["tags"]).downcase }.values.flat_map do |named| - leader = named.to_h { |way| [way["id"], way["id"]] } - find = lambda do |id| - id = leader[id] while leader[id] != id - id - end - first_way = {} - named.each do |way| - way["nodes"].each do |node| - other = first_way[node] ||= way["id"] - joined = [find.(way["id"]), find.(other)] - leader[joined.max] = joined.min - end + connected(named) { |way| way["nodes"] }.filter_map { |trail| path_candidate(trail) } + end + end + + # The named paths, those tiles list as pieces of one, as across a tile's + # edge, joined where they share a way. + def self.joined_paths(paths) + connected(paths) { |path| path[:ways] }.map do |pieces| + next pieces.first if pieces.size == 1 + + ways = pieces.flat_map { |piece| piece[:ways] }.uniq.sort + corners = pieces.map { |piece| piece[:bounds] }.transpose + corners = [corners[0].min, corners[1].min, corners[2].max, corners[3].max] + { id: -ways.first, name: pieces.first[:name], latitude: (corners[0] + corners[2]) / 2.0, + longitude: (corners[1] + corners[3]) / 2.0, bounds: corners, span: distance(*corners).round, + notable: pieces.any? { |piece| piece[:notable] }, ways: ways } + end + end + + # The items, grouped where they share any of the keys the block gives for each. + def self.connected(items) + leader = (0...items.size).to_a + find = lambda do |index| + index = leader[index] while leader[index] != index + index + end + first = {} + items.each_with_index do |item, index| + yield(item).each do |key| + joined = [find.(index), find.(first[key] ||= index)] + leader[joined.max] = joined.min end - named.group_by { |way| find.(way["id"]) }.values.filter_map { |trail| path_candidate(trail) } end + items.each_index.group_by { |index| find.(index) }.values.map { |group| items.values_at(*group) } end def self.path_candidate(ways) @@ -435,6 +470,14 @@ def self.routes(ids, connections, cache, failures: nil, paths: {}) keys = ids.to_h { |id| [id, "#{ROUTE_KEY}#{id}"] } found = cache.read_multi(*keys.values) RouteStore.details(ids.reject { |id| found.key?(keys[id]) }).each { |id, details| found[keys[id]] = details } + # A named path measured from fewer of its ways than are known now is measured again, from them all. + paths = paths.slice(*ids).to_h do |id, ways| + measured = found[keys[id]] + next [id, ways] unless measured.is_a?(Hash) && (ways - Array(measured[:ways])).any? + + found.delete(keys[id]) + [id, (ways | Array(measured[:ways])).sort] + end missing = ids.reject { |id| found.key?(keys[id]) || (id.negative? && !paths.key?(id)) } if missing.any? id_of = keys.invert @@ -698,6 +741,9 @@ def self.elements(query, connections, cache, wait: SLOT_WAIT_SECONDS, timeout: 2 begin connections.each_with_index do |connection, index| return request(connection, query) + rescue SearchErrors::ResponseTooLarge + # The other instance would list as much, so it isn't asked, and isn't preferred. + raise rescue SearchErrors::UpstreamError cache.write(FAILOVER_KEY, !cache.read(FAILOVER_KEY), expires_in: FAILOVER_TTL) if index.zero? raise if index == connections.size - 1 diff --git a/app/services/trails_service.rb b/app/services/trails_service.rb index 7b70581..7e5669b 100644 --- a/app/services/trails_service.rb +++ b/app/services/trails_service.rb @@ -263,9 +263,10 @@ def initialize(station, place, access, result, transit, hiking, elevation, noise @stuck = Concurrent::AtomicBoolean.new end - # Notes the ways of the named paths among the routes the search may check. + # Notes the ways of the named paths among the routes the search may check, + # all those of each that any of its tiles lists. def known(routes) - routes.each { |route| @paths[route[:id]] = route[:ways] if route[:ways] } + routes.each { |route| @paths[route[:id]] = (@paths.fetch(route[:id], []) | route[:ways]).sort if route[:ways] } end def check(ids) diff --git a/config/guides.yml b/config/guides.yml index 6246201..d22534a 100644 --- a/config/guides.yml +++ b/config/guides.yml @@ -3,9 +3,9 @@ # build finds at least GuideService::MIN_HIKES hikes from it. Check that a new # city's hikes are its own: Philadelphia's were nearly all New York's, hours # away, since few hiking routes were mapped near its train stations, when -# searches only counted stops trains reach. Searches use OpenStreetMap's hiking -# route relations, which are rare around Los Angeles and San Francisco, where -# trails are mostly mapped as paths alone. +# searches only counted stops trains reach and OpenStreetMap's hiking route +# relations, before they counted named paths too, as trails are mostly mapped +# around Los Angeles and San Francisco. new-york-city: name: New York City origin: Midtown Manhattan diff --git a/test/services/overpass_service_test.rb b/test/services/overpass_service_test.rb index 09814f7..4aee56d 100644 --- a/test/services/overpass_service_test.rb +++ b/test/services/overpass_service_test.rb @@ -127,13 +127,14 @@ def centered(row, column, minutes) found = routes_in(connection, tiles: [[47.0, -122.0], [47.5, -122.0], [48.5, -122.5]], cache: cache) assert_equal [1, 2], found.pluck(:id).sort - region = "(47.0,-122.5,49.0,-121.5)" boxes = ["(47.0,-122.0,47.5,-121.5)", "(47.5,-122.0,48.0,-121.5)", "(48.5,-122.5,49.0,-122.0)"] - # Named paths come too, but not the ways of hiking routes. - assert_equal %([out:json][timeout:60];relation["type"="route"]["route"="hiking"]#{region}->.region;) + - "(#{boxes.map { |box| "relation.region#{box};" }.join});out tags bb;" \ - "way(r.region)->.routed;way#{OverpassService::PATH_FILTER}#{region}->.paths;" \ - "((#{boxes.map { |box| "way.paths#{box};" }.join}); - .routed;);out body bb;", queries.sole + wider = ["(46.95,-122.05,47.55,-121.45)", "(47.45,-122.05,48.05,-121.45)", "(48.45,-122.55,49.05,-121.95)"] + # Named paths come too, a little beyond the tiles, but not the ways of hiking routes. + assert_equal %([out:json][timeout:60];relation["type"="route"]["route"="hiking"](47.0,-122.5,49.0,-121.5)->.region;) + + "(#{boxes.map { |box| "relation.region#{box};" }.join});out tags bb;" + + %(relation["type"="route"]["route"="hiking"](46.95,-122.55,49.05,-121.45);way(r)->.routed;) + + "way#{OverpassService::PATH_FILTER}(46.95,-122.55,49.05,-121.45)->.paths;" \ + "((#{wider.map { |box| "way.paths#{box};" }.join}); - .routed;);out body bb;", queries.sole # The route across two tiles is in both. assert_equal [2], routes_in(connection, tiles: [[47.5, -122.0]], cache: cache).pluck(:id) assert_equal [1, 2], routes_in(connection, tiles: [[47.5, -122.0], [47.0, -122.0]], cache: cache).pluck(:id).sort @@ -313,13 +314,29 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events end assert_equal 1, calls - calls = 0 - huge = stub_connection(:post, {}) do - calls += 1 + queries = [] + huge = stub_connection(:post, {}) do |request| + queries << URI.decode_www_form(request.body).to_h.fetch("data") raise SearchErrors::ResponseTooLarge end assert_raises(SearchErrors::ResponseTooLarge) { routes_in(huge, cache: ActiveSupport::Cache::MemoryStore.new) } - assert_equal 1, calls + # The other instance would list as much, so only the tile's routes alone are asked for again. + assert_equal [true, false], queries.map { |query| query.include?("out body bb") } + end + + test "where named paths are too many to list with the routes, the tiles keep their routes alone" do + cache = ActiveSupport::Cache::MemoryStore.new + queries = [] + dense = stub_connection(:post, lambda { |request| + queries << URI.decode_www_form(request.body).to_h.fetch("data") + raise SearchErrors::ResponseTooLarge if queries.last.include?("out body bb") + + { "elements" => [candidate_of(route_element(id: 1))] } + }) + assert_equal [1], routes_in(dense, cache: cache).pluck(:id) + assert_equal [1], routes_in(dense, cache: cache).pluck(:id) + assert_equal 2, queries.size + assert_nil cache.read(OverpassService::FAILOVER_KEY) end test "a query that fails quickly on both instances succeeds on the preferred one's second try" do @@ -489,6 +506,38 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events assert_not cache.exist?("#{OverpassService::ROUTE_KEY}-5") end + test "a trail across a tile's edge is one, whichever tiles list its pieces, and measured from all its ways" do + cache = ActiveSupport::Cache::MemoryStore.new + queries = [] + # Way 100 crosses 47.5° north, between way 200 to its south and way 300 to its north. + across = path_way(100, nodes: [1, 2], latitude: 47.48, length: 0.04) + south, north = path_way(200, nodes: [3, 1], latitude: 47.4, length: 0.08), path_way(300, nodes: [2, 4], latitude: 47.52, length: 0.08) + connection = stub_connection(:post, lambda { |request| + query = URI.decode_www_form(request.body).to_h.fetch("data") + queries << query + named = query[/way\(id:([\d,]+)\)->\.named/, 1].to_s.split(",").map(&:to_i) + ways = if query.include?("out ids") then [across, south, north].select { |way| named.include?(way["id"]) } + elsif query.include?("relation.region(47.0,") then [across, south].map { |way| listed_way(way) } + else [across, north].map { |way| listed_way(way) } + end + { "elements" => ways } + }) + found = stub_const(OverpassService, :TILES_PER_QUERY, 1) do + routes_in(connection, tiles: [[47.0, -122.0], [47.5, -122.0]], cache: cache) + end + assert_equal 2, queries.size + assert_equal [[-100, [100, 200, 300]]], found.map { |route| route.values_at(:id, :ways) } + + # Measured from the pieces one tile listed, it's measured again once its other ways are known. + first = OverpassService.trails_for([-100], lat: 47.4, lon: -122.0, connections: [connection], cache: cache, + paths: { -100 => [100, 200] }).sole + whole = OverpassService.trails_for([-100], lat: 47.4, lon: -122.0, connections: [connection], cache: cache, + paths: { -100 => [100, 300] }).sole + assert_includes queries.last, "way(id:100,200,300)->.named;" + assert_equal [100, 200, 300], cache.read("#{OverpassService::ROUTE_KEY}-100")[:ways] + assert_in_delta first.length * 5 / 3, whole.length, 0.01 + end + test "a named path is a little less promising than a hiking route alike" do assert_equal [1, -1], pick([candidate(-1, 10, name: "Cougar Ridge"), candidate(1, 10, name: "Squak Ridge")]) end From 8504af3bc529a0ba0c05cca757a0e7abd81cac2e Mon Sep 17 00:00:00 2001 From: Jeevan Topgi Date: Mon, 5 Oct 2026 20:26:08 -0400 Subject: [PATCH 3/3] Keep a named path's details where measuring it again fails, and join its pieces across all tiles Measuring a path again from ways its tiles list now keeps what was measured when that fails, as Overpass does from the server, and doesn't try again for 6 hours. Details list the ways a path was measured from, those gone since too, so they aren't asked for again and again. Pieces from the first four tiles and the rest are joined too, keeping the id of a piece already checked. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 8 +++-- app/services/overpass_service.rb | 42 +++++++++++++++++--------- app/services/trails_service.rb | 9 +++--- test/services/overpass_service_test.rb | 28 +++++++++++++++++ test/services/trails_service_test.rb | 19 +++++++++--- 5 files changed, 80 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index de48911..e535250 100644 --- a/README.md +++ b/README.md @@ -125,8 +125,12 @@ test Overpass connectivity from your deployment before launching. listed with their nodes but not their geometry (about 3 MB for four tiles around the Issaquah Alps or Marin), and 0.05° beyond the tiles, so a trail across a tile's edge is found whole from either side. Pieces of one that - tiles list apart are joined where they share a way, and a path measured from - fewer of its ways than a search knows is measured again from them all. A + tiles list apart, the first four tiles' and the rest's too, are joined where + they share a way, keeping the id of a piece already checked. A path measured + from fewer of its ways than a search knows is measured again from them all, + or, where that fails, as Overpass does from the server, keeps what was + measured, without trying again for 6 hours; details list the ways a path was + measured from, those gone since too, so they aren't asked for again. A named path's id is the negative of its first way's, its details are measured from its ways once a search picks it, and its highlights found along them, and its OpenStreetMap link is to that way. A mapped hiking route is likelier diff --git a/app/services/overpass_service.rb b/app/services/overpass_service.rb index c3336e4..6a389e8 100644 --- a/app/services/overpass_service.rb +++ b/app/services/overpass_service.rb @@ -45,6 +45,11 @@ module OverpassService # Paths are listed this far beyond their tiles too, so a trail across a # tile's edge is found whole, and the same, from either side. PATH_MARGIN_DEGREES = 0.05 + # A named path measured from fewer of its ways than a search knows is + # measured again, but where that fails, as Overpass does from the server, + # keeps what was measured, and isn't tried again for this long. + REMEASURE_KEY = "overpass:remeasure:v1:".freeze + REMEASURE_PAUSE = 6.hours # A hiking route someone mapped is likelier to be a hike worth the trip than # a named path, so paths are this much less promising. PATH_PROMISE = 0.5 @@ -338,15 +343,17 @@ def self.path_candidates(ways) end # The named paths, those tiles list as pieces of one, as across a tile's - # edge, joined where they share a way. - def self.joined_paths(paths) + # edge, joined where they share a way, keeping the id of any piece in keep, + # as when a search has checked it. + def self.joined_paths(paths, keep: Set.new) connected(paths) { |path| path[:ways] }.map do |pieces| next pieces.first if pieces.size == 1 ways = pieces.flat_map { |piece| piece[:ways] }.uniq.sort corners = pieces.map { |piece| piece[:bounds] }.transpose corners = [corners[0].min, corners[1].min, corners[2].max, corners[3].max] - { id: -ways.first, name: pieces.first[:name], latitude: (corners[0] + corners[2]) / 2.0, + id = pieces.map { |piece| piece[:id] }.find { |each| keep.include?(each) } || -ways.first + { id: id, name: pieces.first[:name], latitude: (corners[0] + corners[2]) / 2.0, longitude: (corners[1] + corners[3]) / 2.0, bounds: corners, span: distance(*corners).round, notable: pieces.any? { |piece| piece[:notable] }, ways: ways } end @@ -470,14 +477,12 @@ def self.routes(ids, connections, cache, failures: nil, paths: {}) keys = ids.to_h { |id| [id, "#{ROUTE_KEY}#{id}"] } found = cache.read_multi(*keys.values) RouteStore.details(ids.reject { |id| found.key?(keys[id]) }).each { |id, details| found[keys[id]] = details } - # A named path measured from fewer of its ways than are known now is measured again, from them all. - paths = paths.slice(*ids).to_h do |id, ways| - measured = found[keys[id]] - next [id, ways] unless measured.is_a?(Hash) && (ways - Array(measured[:ways])).any? - - found.delete(keys[id]) - [id, (ways | Array(measured[:ways])).sort] - end + # Named paths measured from fewer of their ways than are known now are measured again, from them all. + paths = paths.slice(*ids) + stale = paths.select { |id, ways| found[keys[id]].is_a?(Hash) && (ways - Array(found[keys[id]][:ways])).any? } + paused = stale.empty? ? {} : cache.read_multi(*stale.keys.map { |id| "#{REMEASURE_KEY}#{id}" }) + stale = stale.keys.reject { |id| paused.key?("#{REMEASURE_KEY}#{id}") }.to_h { |id| [id, found.delete(keys[id])] } + stale.each { |id, measured| paths[id] = (paths[id] | Array(measured[:ways])).sort } missing = ids.reject { |id| found.key?(keys[id]) || (id.negative? && !paths.key?(id)) } if missing.any? id_of = keys.invert @@ -494,9 +499,15 @@ def self.routes(ids, connections, cache, failures: nil, paths: {}) error = failure {} end) - # Routes another search was looking up, but couldn't in time. - error ||= SearchErrors::ProviderBusy.new(BUSY) unless missing.all? { |id| found.key?(keys[id]) } - if error + stale.each do |id, measured| + next if found.key?(keys[id]) + + found[keys[id]] = measured + cache.write("#{REMEASURE_KEY}#{id}", true, expires_in: REMEASURE_PAUSE) + end + # Routes that couldn't be looked up, or that another search was looking up, but couldn't in time. + if missing.any? { |id| !found.key?(keys[id]) } + error ||= SearchErrors::ProviderBusy.new(BUSY) failures&.push(error) raise error if ids.none? { |id| found.key?(keys[id]) } end @@ -536,7 +547,8 @@ def self.fetch_routes(ids, connections, cache, paths = {}) by_id = with_geometry.index_by { |way| way["id"] } named.each do |id| path = path_attributes(id, paths.fetch(id).filter_map { |way| by_id[way] }, paved) - routes[id] = path if path + # Measured from all of the ways asked for, though some may have gone since their tiles listed them. + routes[id] = path.merge(ways: paths.fetch(id).sort) if path end routes end diff --git a/app/services/trails_service.rb b/app/services/trails_service.rb index 7e5669b..75f6b44 100644 --- a/app/services/trails_service.rb +++ b/app/services/trails_service.rb @@ -233,10 +233,11 @@ def self.from_station(station, departure_time, transit: TransitousService, hikin if farther more = optional { finished(farther).value! } result.complete = false unless more - more = Array(more).reject { |route| relief.key?(route[:id]) } - search.known(more) - relief = relief.merge(reliefs(more, elevation)) - routes = (routes + more).uniq { |route| route[:id] } + # Named paths across the edge between the first tiles and the rest are joined, keeping the ids of those checked. + paths, routes = (routes + Array(more)).partition { |route| route[:ways] } + routes = routes.uniq { |route| route[:id] } + OverpassService.joined_paths(paths, keep: search.checked) + search.known(routes) + relief = relief.merge(reliefs(routes.reject { |route| relief.key?(route[:id]) }, elevation)) end # Counting the routes checked, so the far ones it keeps room for are checked too. search.check(hiking.pick(routes, access: access, relief: relief, checked: search.checked)) diff --git a/test/services/overpass_service_test.rb b/test/services/overpass_service_test.rb index 4aee56d..2baa110 100644 --- a/test/services/overpass_service_test.rb +++ b/test/services/overpass_service_test.rb @@ -538,6 +538,34 @@ def held_connection(queries, holds: {}, started: Hash.new { |events, key| events assert_in_delta first.length * 5 / 3, whole.length, 0.01 end + test "a named path that can't be measured again keeps what was measured, a while, and ways gone since aren't asked for again" do + cache = ActiveSupport::Cache::MemoryStore.new + calls = 0 + down = stub_connection(:post, {}) do + calls += 1 + raise Faraday::ConnectionFailed, "timed out" + end + cache.write("#{OverpassService::ROUTE_KEY}-1", { osm_id: -1, name: "Mount Si Trail", latitude: 47.0, longitude: -122.0, + length: 2.0, path: [[[47.0, -122.0], [47.03, -122.0]]], loop: false, paved: 0.0, ways: [1, 2] }) + failures = [] + measure = -> { OverpassService.trails_for([-1], lat: 47.0, lon: -122.0, connections: [down], cache: cache, failures: failures, paths: { -1 => [1, 2, 3] }) } + assert_equal ["Mount Si Trail"], measure.().map(&:name) + assert_empty failures + asked = calls + assert_equal ["Mount Si Trail"], measure.().map(&:name) + assert_equal asked, calls + + # Measured from ways 1 and 2, though 3 has gone, it isn't measured again for 3. + queries = [] + paths = [path_way(1, nodes: [10, 11]), path_way(2, nodes: [11, 12], latitude: 47.02)] + gone = overpass_connection(routes: [], paths: paths, queries: queries) + 2.times do + OverpassService.trails_for([-7], lat: 47.0, lon: -122.0, connections: [gone], cache: cache, paths: { -7 => [1, 2, 3] }) + end + assert_equal 1, queries.size + assert_equal [1, 2, 3], cache.read("#{OverpassService::ROUTE_KEY}-7")[:ways] + end + test "a named path is a little less promising than a hiking route alike" do assert_equal [1, -1], pick([candidate(-1, 10, name: "Cougar Ridge"), candidate(1, 10, name: "Squak Ridge")]) end diff --git a/test/services/trails_service_test.rb b/test/services/trails_service_test.rb index d32decf..fe3db0e 100644 --- a/test/services/trails_service_test.rb +++ b/test/services/trails_service_test.rb @@ -145,9 +145,9 @@ class FakeHiking # others, as those stored are where Overpass can't be reached, saying so in its failures, until it's been # looked up busy times. def initialize(trails, far: [], highlights: {}, failing: [], unstored: [], busy: Float::INFINITY, release: nil, - stuck: [:highlights], tiles_failing: [], crowded: false) + stuck: [:highlights], tiles_failing: [], crowded: false, ways: {}) @trails, @far, @highlights, @failing, @release, @stuck = trails, far, highlights, failing, release, stuck - @unstored, @busy, @tiles_failing, @crowded = unstored, busy, tiles_failing, crowded + @unstored, @busy, @tiles_failing, @crowded, @ways = unstored, busy, tiles_failing, crowded, ways @batches, @threads, @tile_requests, @reliefs, @checked_before, @paths_given = [], [], Concurrent::Array.new, [], [], [] end @@ -168,10 +168,11 @@ def routes_in(tiles, failures: nil) failures&.push(SearchErrors::UpstreamError.new("busy")) if @tiles_failing.include?(first ? :first_tiles : :other_tiles) - # Named paths, with negative ids, list their ways. + # Named paths, with negative ids, list their ways: their first, unless told others. (first ? @trails : @far).map do |trail| - { id: trail.osm_id, latitude: trail.latitude, longitude: trail.longitude, ways: ([-trail.osm_id] if trail.osm_id.negative?) } - .compact + ways = @ways[trail.osm_id] || ([-trail.osm_id] if trail.osm_id.negative?) + { id: trail.osm_id, latitude: trail.latitude, longitude: trail.longitude, bounds: [trail.latitude, -122.1, trail.latitude, -122.1], + ways: ways }.compact end end @@ -331,6 +332,14 @@ def minutes(count) assert_equal [{ -77 => [77] }], hiking.paths_given end + test "pieces of a named path in the first tiles and the rest are one, which isn't checked again" do + near, far = trail("Ridge Trail", osm_id: -100), trail("Ridge Trail", osm_id: -50) + hiking = FakeHiking.new([near], far: [far], ways: { -100 => [100, 200], -50 => [50, 100] }) + result = search(transit: FakeTransit.new(trips: { "Ridge Trail" => minutes(30) }), hiking: hiking) + assert_equal [[-100]], hiking.batches + assert_equal ["Ridge Trail"], result.trails.map(&:name) + end + test "tiles in bands of travel time with more than their share are picked knowing how far the land rises across them" do hiking = FakeHiking.new([trail("a")], crowded: true) elevation = FakeElevation.new(relief: { [47.5, -122.5] => 300 })