.. _config_http_route_specifiers_dynamic_modules: Dynamic modules route specifier =============================== Overview -------- The :ref:`DynamicModuleRouteSpecifier ` configuration specifies a :ref:`route specifier ` backed by a :ref:`dynamic module `. For each request the module keeps the route that route matching resolved, refines it, replaces it with one of the route templates it declares, drops it, or lets route matching carry on with the next route. The module is invoked while the route is being resolved, and again whenever the route is recomputed, so it must be able to reach a decision from the request and the stream info alone. The call is synchronous and cannot be time boxed, so a module must not block or perform I/O. Route templates --------------- A module cannot build a route out of thin air. Instead it selects one of the :ref:`route_templates ` the specifier declares. Each template is an ordinary :ref:`Route ` that is built and validated once, when the specifier is configured, so an invalid template is rejected at configuration load rather than on the request path. A template inherits from the virtual host and the route configuration the specifier is configured on, exactly like a configured route does. When the module selects a template, Envoy evaluates it against the request like a configured route. The :ref:`match ` must hold, and the action is resolved the usual way, including :ref:`weighted_clusters ` and :ref:`cluster specifier plugins `. A match that does not hold is a failure, because the rewrites of the action are only correct for a request the match accepts. Decisions --------- The module records one of five decisions with ``set_decision``: * ``Unspecified`` is the default, in effect when the module records no decision: Envoy generates a new route based on what the setter callbacks recorded, from the selected template, or from the route the specifier was given when no template was selected, with the recorded properties applied on top. With nothing recorded it uses the route the specifier was given, unchanged. * ``PassThrough`` uses the route the specifier was given, unchanged, ignoring whatever the module recorded. * ``NoRoute`` uses no route, so the request is handled as if nothing had matched. * ``Error`` reports that the module could not decide. * ``ReusePrevious`` uses the previous route of the stream unchanged, without building a new route. The hook returns a status rather than the decision: ``Continue`` hands the produced route to the route specifiers configured after this one, ``StopIteration`` makes it the final route, and ``StopIterationAndSkipRoute`` drops the route and lets route matching carry on with the next route, ignoring the recorded decision. A decision Envoy cannot honor is handled by the configured :ref:`failure_policy `, which passes the request through to the route table, drops the route, or lets route matching carry on with the next route. The SDK reports an error when the module panics, so a panic is handled by the same policy rather than crashing Envoy. Stream consistent decisions --------------------------- Envoy resolves the route more than once per stream, for example after a filter clears the route cache. On a later resolution the module reads the route the connection manager last installed with ``get_previous_route`` and ``get_previous_route_metadata``, which are null on the first resolution, after an internal redirect, and when a filter installed a null route. Reading a marker the module wrote into the metadata of a route it built earlier, through ``set_route_metadata_string``, is how a module keeps a stream on the version it first served without depending on route object identity. ``ReusePrevious`` uses that previous route unchanged and builds nothing. When the stream has no previous route the decision is rejected and counted in ``reuse_previous_rejected``, and the failure policy applies. A module that builds routes from versioned state can lease that version for as long as any route it built is alive. ``set_route_user_data`` records a ``u64`` on the produced route, which forces the route to be wrapped even with no other override, and :ref:`on_route_specifier_route_destroy ` fires with that value when the route is destroyed. The hook may run on any thread and after the stream is gone, since another owner may retain the route, so a module must treat it as a lease release only. A decision that builds no route to carry the value hands it instead to :ref:`on_route_specifier_user_data_discard `, ``on_user_data_discard`` in the Rust SDK. This happens for a decision other than ``Unspecified``, a skipped route, a decision Envoy cannot honor and a ``Hit`` from ``lookup_built_route``. The hook runs on the thread that resolves the route, before the specifier returns. With both hooks exported, a module gets back the last value it records for each decision. The :ref:`specifier_instance_id ` lets two specifier instances sharing one module tell their routes apart. Shadowing a routing change -------------------------- A module receives the route that route matching resolved through its context, so it can shadow a routing change on its own without any support from Envoy. In a dry run the module computes the route it would ask for, compares it against the resolved route, counts the outcome on its own metrics, and records ``PassThrough`` so that routing stays unchanged. Once the counts give confidence, the same module records its overrides or selects its template to apply the decision. The ``route_specifier_shadow.rs`` test module under ``test/extensions/dynamic_modules/test_data/rust`` shows both runs, comparing the cluster name. Migrating routing to a module ----------------------------- A module can take over the routing of a virtual host one step at a time, while the route table it replaces stays in place as a fallback. The pattern is a catch all route placed first in the route table, carrying the specifier at the route level. The catch all route matches every request, so the specifier runs first for each one. For a request the module owns it records its overrides or selects its template, and for a request the module leaves to the route table it returns the ``StopIterationAndSkipRoute`` status, which drops the catch all route and lets route matching carry on with the routes below it. Setting :ref:`failure_policy ` to ``CONTINUE_MATCHING`` makes a module failure fall back to the route table as well, rather than to the catch all route's own action. When the module owns the whole virtual host, the routes below the catch all route can be removed and the ``failure_policy`` changed to ``NO_ROUTE``. Route overrides --------------- The properties that are built from other extensions, such as the retry policy and the request mirroring policies, are declared as :ref:`route_overrides `. Each override is built and validated once when the specifier is configured, and the module selects one by ``override_id``. An override that replaces no property is rejected, so a module can rely on a declared override changing something. An override may also carry ``tracing`` and ``metadata``, which are route level properties valid on any route including a direct response or a redirect, unlike the retry policy and the other route entry properties. Module selected route metadata can change authorization decisions, rate limiting descriptors and access log fields, so the override set is static configuration chosen by trusted in process code. Rewriting the path ------------------ ``set_prefix_rewrite`` records the path a prefix rewrite produces for the current request, so the route it produces is specific to that request. ``set_prefix_rewrite_rule`` records the rewrite as a rule instead, which Envoy applies to the path the request carries when the request headers are finalized, the way a configured route applies its :ref:`prefix_rewrite `. A rule replaces a literal prefix of the path without its query string, or all of that path, with a replacement that starts with a slash, and it is checked against the request the decision is taken for. Envoy takes the path sent upstream from the first of these that yields one, whatever the call order. * The path recorded with ``set_path`` or ``set_prefix_rewrite``. * The rule recorded with ``set_prefix_rewrite_rule``. * The regex rewrite of the selected route override. * The rewrites of the route. The regex rewrite applies when the decision resolves, or when the request headers are finalized once a rule is recorded or the built route cache looks the decision up, and like a configured regex rewrite it yields the path unchanged when its pattern does not match. A rewrite applied when the request headers are finalized records ``x-envoy-original-path`` the way a configured route does, replacing one the request carries. A request for which neither the rule nor the regex rewrite after it yields a path at that point, for example because a filter changed the path without clearing the route cache, is counted in ``rewrite_not_applied``. Caching built routes -------------------- A module whose routes are a function of a small key can let each worker cache them with :ref:`built_route_cache `. After selecting a template, the module calls ``lookup_built_route`` with a key. On a hit Envoy uses the route a previous request on the worker built for the key and the template, and the module records nothing more, though a decision or a skip it records still wins over the hit. On a miss the module records the route as usual, and Envoy caches the route the decision builds unless a route is already cached for the key. A module whose routes rely on the cache can reject a configuration without one from its configuration hook, through ``config_has_built_route_cache``, ``has_built_route_cache`` in the Rust SDK. The route built for a key serves later requests, so it must be a function of the key. The key covers whatever the module reads to decide, including the route it runs for, since one cache serves every route of a virtual host that shares the specifier. It identifies the route exactly, since a collision serves the route of one key for the other, and a module whose routes change over time folds a version of what they depend on into it. A route configuration update starts with empty caches, and releases the cached routes of the replaced configuration once no stream uses it. Envoy keeps the dependencies it introduces itself out of the cache. A cached route that carries a rule recorded with ``set_prefix_rewrite_rule`` serves only the paths the rule applies to. A recording that calls ``set_prefix_rewrite``, or for which ``set_prefix_rewrite_rule`` rejected a rule or replaced one it accepted, only serves the current request. A template whose cluster is resolved per request, such as one with weighted clusters, a cluster header or a cluster specifier plugin, is not cached. A worker evicts a route it has not used recently when it is full, and a route idle for longer than ``max_idle_time``. A hit is verified with a chance of one in ``verify_one_in``, for which ``lookup_built_route`` returns ``HitVerify``. The module then records the route exactly as on a miss and Envoy uses the route the recording builds. When the decision fails or its recording depends on the path of the request through Envoy or differs from the one the cached route was built from, Envoy counts ``built_route_cache_verify_mismatch`` and evicts the cached route. A ``Hit`` ignores the overrides the module records and hands its user data to ``on_route_specifier_user_data_discard``. A route that carries user data fires ``on_route_specifier_route_destroy`` once neither the cache nor a stream holds it. Notes ----- * The request headers are read-only. A recorded path, authority or header mutation is applied through the header transforms of the route the decision produces, so that Envoy applies it at the right point of the request lifetime. * ``get_cluster_host_count`` reports whether a cluster is routable from the current worker and returns host counts at a priority level. It uses ``getThreadLocalCluster()``, so it can return false even when the cluster is configured but not yet warmed on the worker. * Custom counters, gauges and histograms can be defined during configuration and recorded during resolution, and are emitted under the ``metrics_namespace`` prefix of ``DynamicModuleConfig``. Statistics ---------- The specifier emits statistics rooted at ``.route_specifier..``, where ``stat_prefix`` is the :ref:`stat_prefix ` of the specifier, sharing the ``metrics_namespace`` of the module-defined metrics above. .. csv-table:: :header: Name, Type, Description :widths: 1, 1, 2 decision_pass_through, Counter, Requests for which the module kept the resolved route. decision_has_override, Counter, Requests the default decision resolved from the route the specifier was given with the recorded overrides applied. decision_has_template, Counter, Requests for which the module selected a route template. decision_no_route, Counter, Requests for which the module dropped the route. decision_error, Counter, Requests for which the module could not decide. decision_reuse_previous, Counter, Requests for which the module reused the previous route. route_skipped, Counter, Requests for which the module let route matching carry on with the next route. unknown_status, Counter, Requests for which the module returned a status Envoy does not know. runtime_skipped, Counter, Requests outside ``runtime_fraction``. failure_module_error, Counter, Decisions not honored because the module reported an error. failure_template_not_selected, Counter, Decisions not honored because a template selection named an identifier that is not declared. failure_template_match_failed, Counter, Decisions not honored because the match of the selected template did not hold. failure_override_without_route, Counter, Decisions not honored because there was no route to refine. failure_override_on_non_route_entry, Counter, Decisions not honored because route entry properties were recorded for a direct response. failure_route_metadata, Counter, Decisions not honored because a typed metadata factory rejected the recorded metadata. reuse_previous_rejected, Counter, ReusePrevious decisions rejected because the stream had no previous route. route_destroy, Counter, Routes built with user data that were destroyed and fired the route destroy hook. user_data_discard, Counter, User data that no route carried and that the user data discard hook received. rewrite_not_applied, Counter, Requests for which neither the prefix rewrite rule nor the route override regex rewrite yielded a path when the request headers were finalized. built_route_cache_disabled, Counter, Built route cache lookups that returned ``Disabled``. built_route_cache_hit, Counter, Built route cache lookups that found a route. built_route_cache_miss, Counter, Built route cache lookups that found no route for the request. built_route_cache_verify, Counter, Built route cache hits verified against what the module recorded. built_route_cache_verify_mismatch, Counter, Verified hits that evicted the cached route. built_route_cache_evict, Counter, Cached routes evicted because a worker was full or the route was idle for longer than ``max_idle_time``. built_route_cache_entries, Gauge, Routes cached by the workers of the specifiers that share the stat prefix. on_route_duration, Histogram, Time in microseconds the module spent deciding. specifier_duration, Histogram, Time in microseconds the specifier spent on a request that ``runtime_fraction`` does not skip. Configuration ------------- * This extension should be configured with the type URL ``type.googleapis.com/envoy.extensions.router.route_specifiers.dynamic_modules.v3.DynamicModuleRouteSpecifier``. * :ref:`v3 API reference ` .. attention:: Dynamic modules run in-process with the same privileges as Envoy. Only load modules you trust. This extension is currently under active development. Capabilities and ABI are expected to evolve. Configuration example --------------------- .. code-block:: yaml route_config: virtual_hosts: - name: default domains: ["*"] route_specifiers: - name: envoy.router.route_specifiers.dynamic_modules typed_config: "@type": type.googleapis.com/envoy.extensions.router.route_specifiers.dynamic_modules.v3.DynamicModuleRouteSpecifier dynamic_module_config: name: my_route_specifier do_not_close: true specifier_name: my_specifier_impl stat_prefix: my_specifier failure_policy: PASS_THROUGH route_templates: - template_id: canary route: match: prefix: "/" route: cluster: canary_service timeout: 5s routes: - match: prefix: "/" route: cluster: web_service