Specter Solves Clojure Nested Data Bottleneck via Path Navigation
Misophistful
Specter eliminates multi-line boilerplate and preserves collection types by playing back navigation paths in reverse to update nested values in place.
AI-assisted summary of Understanding Specter: Clojure's missing piece — original source. Check the original for full context.
Source published:
Key ideas
- Specter addresses Clojure's tendency to strip collection types during transformations by automatically preserving the original data structure (e.g., sets remain sets, vectors remain vectors) without requiring custom conversion logic. (1:21)
- Unlike standard Clojure core functions, which struggle with nested data and require verbose boilerplate, Specter uses composable path navigators to query, filter, and transform deep structures in one to three lines of code. (3:28)
- Specter's core mechanism operates via a two-phase pipeline: it records a navigation path down to targeted elements, applies the transformation, and replays the path backwards to reconstruct the surrounding data structure. (11:41)
- The library supports advanced navigation capabilities including positional insertion (BEFORE-INDEX, AFTER-ELEM), recursive tree extraction (WALKER), and bi-directional transformations on serialized types like strings via parser pairs. (27:37)
Detailed notes
## Background "Understanding Specter: Clojure's missing piece" was published on YouTube by Misophistful on October 6, 2017. The tutorial screencast aims to teach Clojure developers how to elegantly manipulate nested data structures using Nathan Marz's Specter library.
## TL;DR The video examines two major friction points in standard Clojure—collection type preservation and clumsy nested data manipulation—and demonstrates how Specter resolves them through its path navigation abstraction. The presenter explains Specter's two-phase navigation and playback reconstruction pipeline, contrasting it with core primitives like get-in. Through interactive REPL examples, the screencast explores queries and updates using transform, select, and setval, walks through positional insertions with navigators like BEFORE-INDEX, and demonstrates arbitrary tree searches with WALKER. Finally, the session covers bi-directional string parsing and transformations using paired parse and unparse functions alongside the parser navigator.
## Atmosphere & style The presentation carries the warm, practical tone of an enthusiastic engineer sharing a favorite tool during a live REPL coding session. The host maintains an approachable, unpretentious demeanor, candidly admitting when deep nesting makes navigation tedious and openly admiring the elegance of Specter's underlying design. Watching the tutorial feels exploratory and satisfying, as tangled boilerplate Clojure code is repeatedly distilled into concise, expressive navigation paths.
## Bulletin - [01:21](https://www.youtube.com/watch?v=rh5J4vacG98&t=81s) Specter addresses Clojure's tendency to strip collection types during transformations by automatically preserving the original data structure (e.g., sets remain sets, vectors remain vectors) without requiring custom conversion logic. - [03:28](https://www.youtube.com/watch?v=rh5J4vacG98&t=208s) Unlike standard Clojure core functions, which struggle with nested data and require verbose boilerplate, Specter uses composable path navigators to query, filter, and transform deep structures in one to three lines of code. - [11:41](https://www.youtube.com/watch?v=rh5J4vacG98&t=701s) Specter's core mechanism operates via a two-phase pipeline: it records a navigation path down to targeted elements, applies the transformation, and replays the path backwards to reconstruct the surrounding data structure. - [27:37](https://www.youtube.com/watch?v=rh5J4vacG98&t=1657s) The library supports advanced navigation capabilities including positional insertion (BEFORE-INDEX, AFTER-ELEM), recursive tree extraction (WALKER), and bi-directional transformations on serialized types like strings via parser pairs.
## Key takeaways - [01:21](https://www.youtube.com/watch?v=rh5J4vacG98&t=81s) Standard Clojure core functions like `map` discard concrete collection types in favor of lazy sequences, requiring manual workarounds like `mapv` that still fail to generically preserve types like sets. - [03:28](https://www.youtube.com/watch?v=rh5J4vacG98&t=208s) Manipulating nested data structures in idiomatic Clojure requires awkward boilerplate involving manual destructuring, filtering, mapping, and repacking with `into`. - [07:58](https://www.youtube.com/watch?v=rh5J4vacG98&t=478s) Specter generalizes the path lookup concept found in Clojure's core `get-in`, elevating simple key lookups into a full-featured navigation and querying language. - [11:41](https://www.youtube.com/watch?v=rh5J4vacG98&t=701s) The `transform` function navigates down to specific nested values, applies an update function, and reconstructs the enclosing collections via reverse playback while preserving untouched values. - [15:36](https://www.youtube.com/watch?v=rh5J4vacG98&t=936s) The `select` macro enables read-only queries against deeply nested structures, allowing developers to inspect data at intermediate navigational paths without mutating it. - [15:36](https://www.youtube.com/watch?v=rh5J4vacG98&t=936s) When building Specter navigation paths, developers must carefully track whether a navigator points to an individual collection element or the collection sequence itself. - [23:23](https://www.youtube.com/watch?v=rh5J4vacG98&t=1403s) The `setval` macro provides a clean convenience alternative to `transform`, replacing targeted elements with static values without wrapping them in `constantly`. - [27:37](https://www.youtube.com/watch?v=rh5J4vacG98&t=1657s) Specter allows positional navigation into empty slots between or around sequence items using navigators like `BEFORE-ELEM`, `AFTER-ELEM`, and `BEFORE-INDEX`. - [32:44](https://www.youtube.com/watch?v=rh5J4vacG98&t=1964s) The `WALKER` navigator recursively traverses arbitrary tree structures, pulling out leaves matching specified predicates without requiring explicit structural paths. - [34:24](https://www.youtube.com/watch?v=rh5J4vacG98&t=2064s) Intra-value transformations, such as modifying substring components within dates or times, are handled losslessly through bi-directional `parser` navigators requiring matching `parse` and `unparse` functions.
## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=rh5J4vacG98&t=0s) Introduction to Specter and Screencast Overview
The screencast introduces Specter, a data manipulation library for Clojure authored by Nathan Marz. The presenter expresses deep admiration for the project, noting that it is not only a personal favorite used constantly, but also stands out in broader software engineering terms. As the speaker puts it: “I want to take a little bit of your time to explain why it's one of my favorite Clojure libraries, why I use it all the time, and even why I think it's one of the best pieces of software engineering ever produced.”
The session is organized into three distinct parts: 1. Explaining the motivation behind Specter and why Clojure developers should take an interest in it. 2. Unpacking how Specter works conceptually under the hood. 3. Demonstrating concrete code examples to illustrate its utility and expressive power.
The presenter sets expectations regarding the audience: the talk is aimed primarily at developers who already have a working knowledge of Clojure but have not yet explored Specter deeply. Specter experts might find much of the foundational material already familiar, while complete Clojure beginners might prefer a gentler introductory resource before tackling the advanced patterns shown here.
[01:21](https://www.youtube.com/watch?v=rh5J4vacG98&t=81s) The First Weakness of Clojure: Type Preservation in Collections
While the presenter views Clojure as an exceptionally well-designed language with very few flaws, there are two specific areas where daily development becomes noticeably painful. The first major friction point is Clojure's struggle to maintain original collection types when using standard higher-order functions like `map` or `filter`. When a developer passes a vector of numbers into `map`, the expression evaluates to a lazy sequence (rendered as a list) rather than returning the vector originally provided. Although there are deliberate historical design rationales from Rich Hickey for returning lazy sequences, in typical application logic developers frequently expect an input vector to produce an output vector without stripping away structural identity.
Standard Clojure offers partial workarounds, but they lack generic applicability. Using `mapv` guarantees a vector output, yet it forces the function to assume a vector type explicitly. If a caller passes a set into `mapv`, the output is still coerced into a vector rather than preserving the set. Handling this generically across multiple data structures inevitably forces the programmer to write conditionals or custom conversion logic. Specter solves this out of the box using its core `transform` function. By passing a navigator (such as `ALL`), a transformation function like `inc`, and an input collection (such as a set `#{1 2 3 4}` or a vector), Specter automatically retains and returns the exact data structure type supplied without extra configuration.
```mermaid flowchart TD accTitle: Collection Type Handling Comparison accDescr: Comparing standard Clojure mapping behavior with Specter transform behavior across data types.
subgraph Core["Standard Clojure Core"] V1["Vector: [1 2 3]"] -->|map inc| S1["LazySeq: (2 3 4)"] S2["Set: #{1 2 3}"] -->|mapv inc| V2["Vector: [2 3 4]"] end
subgraph Spec["Specter transform"] V3["Vector: [1 2 3]"] -->|ALL, inc| V4["Vector: [2 3 4]"] S3["Set: #{1 2 3}"] -->|ALL, inc| S4["Set: #{2 3 4}"] end ```
[03:28](https://www.youtube.com/watch?v=rh5J4vacG98&t=208s) The Second Weakness: Manipulating Nested Data Structures
Type retention is merely a helpful auxiliary benefit; Specter's primary purpose is resolving Clojure's clumsy handling of nested data structures. Because Clojure's syntax makes constructing deeply nested maps and vectors effortless, nested data appears everywhere in real-world codebases. However, the standard library is only well-suited for unnested collections. The speaker observes: “Unfortunately the default syntax and core library in Clojure is not really elegant at handling nested data structures; it's really good at dealing with unnested data structures, but as soon as you even put in some kind of basic nesting, suddenly it becomes a nightmare to maintain the structure and to traverse over it.”
To illustrate this shortcoming, the presenter constructs a minimal nested data structure consisting of a vector containing three hash maps:
```clojure [{:a 1:b 2} {:c 3:d 4} {:e 5:f 6}] ```
The objective is straightforward: traverse the data structure and increment only the even numbers while leaving odd numbers and the surrounding structure completely untouched. Accomplishing this with pure Clojure requires building boilerplate helper functions: - An `apply-function-to-hash-map` function that takes a function and a map, runs a `for` comprehension destructuring `[k v]` pairs, maps `(f v)` onto the values, and repacks the sequence into a hash map via `(into {}...)`. - A conditional helper function like `inc-even` taking `n` and evaluating `(if (even? n) (inc n) n)`. - A wrapping call that maps `apply-function-to-hash-map` over the vector, switching from `map` to `mapv` to preserve the outer vector rather than returning a lazy sequence.
This standard implementation quickly balloons into multi-line, fragile, and hard-to-read code just to modify values one level deep inside maps.
[07:58](https://www.youtube.com/watch?v=rh5J4vacG98&t=478s) The Specter Navigation Abstraction and `get-in` Comparison
By contrast, the presenter demonstrates the equivalent operation written in Specter:
```clojure (transform [ALL MAP-VALS even?] inc data) ```
In a single concise expression, Specter navigates through each map in the vector (`ALL`), inspects every value within those maps (`MAP-VALS`), filters exclusively for the even numbers (`even?`), and applies `inc` to them. It yields the exact modified nested vector without requiring any custom destructuring, repacking, or intermediate helper functions.
The presenter acknowledges that the path navigation paradigm can initially feel foreign to Clojure developers accustomed to standard sequencing and threading macros: “I know for me it took me a while to grok it, to understand it, but once you do get it, you realize how simple and powerful a way of describing navigating around the data it really is, and you really fall in love with it.”
To make this concept intuitive, the presenter anchors it in a core Clojure function that developers already know well: `get-in`. When extracting a nested value from nested maps—such as finding the value under key `:c` in `{:a {:b {:c 1}}}`—a developer passes a path vector `[:a:b:c]` to `get-in`. Specter generalizes this exact concept of a path, elevating it from simple key lookups into a full-featured navigation language capable of arbitrary traversals, conditional queries, and updates across nested structures.
[10:40](https://www.youtube.com/watch?v=rh5J4vacG98&t=640s) Basic Navigators vs. Clojure's Core `get-in`
The speaker begins by verifying that standard Clojure access functions behave as expected on nested maps. A lookup returning `1` serves as a baseline:
```clojure ;; Conceptual equivalent of navigating nested structures in core Clojure (get-in [{:b {:c 1}}:foo] [0:b:c]) ;; => 1 ```
The speaker points out that while `get-in` feels like a primitive ancestor of Specter's navigator concept, it remains strictly limited in scope:
> “This here is you can think that is a very basic version of Specter's navigator concept this navigates you through the nested maps but with `get-in` it only works with the keyword and I think it also works with a sequence...”
Testing this in the REPL, the speaker confirms that `get-in` accepts sequential indexing alongside keyword lookups, correctly pulling out `1` when passing a path like `[0:b:c]`. However, this native mechanism cannot flexibly express complex navigation patterns across arbitrary combinations of collections, filtering predicates, and structural transformations in the way Specter does.
[11:41](https://www.youtube.com/watch?v=rh5J4vacG98&t=701s) How `transform` Navigates and Reconstructs Data
To demonstrate Specter's internals, the speaker pastes a previously prepared example using `transform` on a vector of maps:
```clojure ;; Initial structure: [{:a 1:b 2} {:c 3} {:d 4}]
;; Specter transformation: (transform [ALL MAP-VALS even?] inc data) ```
The speaker steps through each phase of navigation to explain how the data structure is traversed. Specter strips away the enclosing vector using `ALL`, targeting all three maps simultaneously while keeping them distinct. Next, `MAP-VALS` navigates to all the map values across those maps—producing the individual scalar values `1`, `2`, `3`, and `4`. Applying the predicate `even?` filters that target set down strictly to `2` and `4`. At the end of this navigation path, Specter runs the transformation function `inc`, incrementing `2` to `3` and `4` to `5`.
```mermaid flowchart TD accTitle: Specter Transform and Reconstruction Flow accDescr: Shows navigation down to target values, transformation, and reconstruction back up.
subgraph Navigation ["1. Path Navigation Down"] A["'[{:a 1:b 2} {:c 3} {:d 4}]'"] -->|"ALL"| B["Maps: '{:a 1:b 2}', '{:c 3}', '{:d 4}'"] B -->|"MAP-VALS"| C["Values: 1, 2, 3, 4"] C -->|"even?"| D["Filtered Targets: 2, 4"] end
subgraph Transform ["2. Transformation"] D -->|"inc"| E["Updated Values: 3, 5"] end
subgraph Reconstruction ["3. Playback Reconstruction Up"] E -->|"Re-merge with odds"| F["Values: 1, 3, 3, 5"] F -->|"Rebuild Maps"| G["Maps: '{:a 1:b 3}', '{:c 3}', '{:d 5}'"] G -->|"Rebuild Vector"| H["'[{:a 1:b 3} {:c 3} {:d 5}]'"] end ```
The speaker highlights the subsequent "reconstruction phase" as the true ingenuity of the library:
> “And then he goes into its reconstruction phase and this is where the real magic happens and this is where I'm not 100% sure on how it works because I've not actually looked into the code it's more from my understanding from Nathan's videos and reading around but what I think happens is this: what it does is it just plays what it recorded in terms of the path backwards to reconstruct back to where you were but along the way it replaces the values that you've updated with your transformation function.”
By playing back the navigation trail in reverse, Specter re-associates the updated values (`3` and `5`) alongside the untouched filtered-out values (`1` and `3`), reconstructs the maps, and finally reassembles the outer vector. Beyond its conceptual cleanliness, the speaker praises Nathan Marz's engineering, noting that hand-rolled Clojure code routinely struggles to match Specter's execution speed.
[15:36](https://www.youtube.com/watch?v=rh5J4vacG98&t=936s) Querying Complex Nested Data with `select`
Specter's path navigators are not restricted to data transformation; by stopping at the end of a navigation path without providing an update function, they act as query mechanisms using `select`. To demonstrate this, the speaker imports a realistic, deeply nested JSON payload that has been converted into Clojure data: a root map containing a `:res` key mapped to a vector of maps, which contain nested `:catalysts` vectors of maps, which in turn contain `:points` vectors containing `:start`, `:stop`, and `:point` entries.
The speaker points out that trying to query or reshape this structure using standard Clojure functions would result in deeply nested, convoluted `map` and `reduce` pipelines. Using Specter, querying for specific properties is straightforward:
```clojure ;; Initial attempt querying all points (select [:res ALL:catalysts ALL:points] data) ```
To restrict the result to only the `:start` and `:stop` keys from each map, the speaker first attempts to append `(submap [:start:stop])` directly after `:points`:
```clojure ;; Fails because:points yields a vector/sequence of maps, not a single map element (select [:res ALL:catalysts ALL:points (submap [:start:stop])] data) ```
The query fails because `:points` points to a vector of elements rather than an individual map element. The speaker explains the critical rule when constructing Specter paths:
> “...and this is what I was talking about before about you need to keep track of are you navigating to an element or to a sequence of elements and because points is actually a sequence of maps that was fine before when we navigated there cuz it just returned the sequence as part of the result but now we want to do a `submap` on all of those maps...”
By inserting `ALL` after `:points` to navigate down into each individual map within the vector—`[:res ALL:catalysts ALL:points ALL (submap [:start:stop])]`—the query evaluates successfully, isolating just the start and stop times across the entire nested structure.
[19:37](https://www.youtube.com/watch?v=rh5J4vacG98&t=1177s) Transforming Deeply Nested Structures: Reversing by `:stop` Time
Moving from read-only queries back to transformation, the speaker shows how the same deep path navigation can update data deep within the hierarchy without disturbing unrelated keys. The goal is to sort each list of `:points` in reverse order based on their `:stop` timestamp.
The speaker constructs the transformation path to target the `:points` vectors:
```clojure (transform [:res ALL:catalysts ALL:points] (comp reverse (partial sort-by:stop)) data) ```
The speaker initially runs `(sort-by:stop)` alone, inspects the REPL output, and notes that because the sample data happened to already be sorted chronologically (e.g., `13:35` followed by `14:35`), no visible change took place. To make the effect obvious, the speaker changes the transformation function to reverse the sort order. Drilling down into the resulting data structure under `:catalysts` and `:points`, the speaker verifies the change:
> “...now hopefully there will be reversed if we drill all the way down again the catalyst cat points yes they've swapped over see see now point two comes first and point one comes second okay so let's go back and figure out what we've actually done here...”
Because the path ended at `:points` (without an extra `ALL`), the transformation function received the entire sequence of point maps at once, enabling it to reorder the sequence in place before Specter reconstructed the surrounding hierarchy intact.
[22:09](https://www.youtube.com/watch?v=rh5J4vacG98&t=1329s) Reversing Vectors and the Expressiveness of Specter
Tracing the flow of the preceding transformation, Specter reaches into the nested structure step-by-step. It navigates down to the target data structure—a vector of maps representing points—and passes that entire vector directly into the transformation function. Inside that function, the data is sorted by the `:stop` time key and then reversed. The updated collection slots cleanly back into the original data layout with all surrounding contexts intact.
```mermaid flowchart LR accTitle: Vector Transformation Flow accDescr: Illustrates Specter navigating to a vector of points, applying sort and reverse, and returning the updated structure. subgraph Navigation ["Specter Navigation"] A["Root Data"] --> B["Nested Path"] B --> C["Target: Vector of Point Maps"] end subgraph Transformation ["Transform Function"] C --> D["sort-by:stop"] D --> E["reverse"] end subgraph Result ["Reconstruction"] E --> F["Replaced in-place"] end ```
The speaker emphasizes how concise and maintainable this paradigm becomes once the syntax clicks. Because you only need to declare where you are navigating and what function you want to apply, complex structural mutations take just one to three lines of code. As the speaker puts it: “As you can see you can do something incredibly powerful with very little code that once you get used to reading it is really highly readable because the these navigations once you get used to the syntax and how it works on what you're looking for it doesn't take long at all to realize how you've navigated through the structure.”
- The transformation function receives the target vector, runs `sort-by` on `:stop`, applies `reverse`, and returns the modified vector to the original position. - Specter leaves all non-targeted surrounding data untouched. - Operations that typically take dozens of lines of nested Clojure manipulation reduce to one to three lines.
---
[23:23](https://www.youtube.com/watch?v=rh5J4vacG98&t=1403s) Replacing Values Directly with `setval`
While `transform` shines when an existing value needs to be passed to a function to compute a replacement, many data manipulation tasks simply involve overwriting a target with a static value. If one were to use `transform` for this, it would require supplying an anonymous function or using `constantly` to ignore the incoming parameter (e.g., `(transform [:timestamp] (constantly 12345) data)`). To make this cleaner, Specter author Nathan Marz provided the helper macro `setval`.
To demonstrate, the speaker targets the `:timestamp` key at the top level of the data structure. Using `(setval:timestamp 123456 data)`, the timestamp updates immediately while leaving the rest of the data structure untouched. The speaker contrasts this with `transform`, illustrating that `setval` acts as a direct convenience function that removes the need for `constantly`: “What `setval` does it's it's just the convenience function where you don't need to put constantly you can just put the value that you want to update it to and that works well for a single value like that but it also works really well you have a collection of values.”
---
[25:19](https://www.youtube.com/watch?v=rh5J4vacG98&t=1519s) Navigating Paths with `FIRST` and Overwriting Map Elements
The speaker shifts from setting top-level keys to targeting specific elements deep within nested sequences using built-in navigators like `FIRST`. Rather than operating across every element with `ALL`, the task is to navigate specifically to the first group within `:res`, down to the first item inside `:catalysts`, and then into `:points` to overwrite its first point map.
```mermaid flowchart TD accTitle: Targeted Element Overwrite accDescr: Step-by-step navigation down to the first point of the first catalyst of the first group to replace it. R[":res"] --> G1["FIRST (Group 1)"] G1 --> C[":catalysts"] C --> Cat1["FIRST (Catalyst 1)"] Cat1 --> P[":points"] P --> P1["FIRST (Point 1)"] P1 -->|"setval {:point:x}"| NewP["Replaced with Point X"] ```
The resulting Specter path uses `[:res FIRST:catalysts FIRST:points FIRST]`. When paired with `setval` and a new map—such as `{:point "X"}`—Specter traverses straight to the first point map and substitutes the new map in place of the original `point-1` start and stop entries. Because the navigator stopped at an individual element, replacing it with a new map cleanly replaces that single element rather than splicing or reshaping the parent vector.
- Navigators can drill down selectively: `[:res FIRST:catalysts FIRST:points FIRST]`. - Supplying a new map `{:point "X"}` via `setval` replaces only that single element. - The sibling points and all surrounding metadata in the catalyst remain preserved.
---
[27:37](https://www.youtube.com/watch?v=rh5J4vacG98&t=1657s) Inserting into Positions: `BEFORE-ELEM`, `AFTER-ELEM`, and `BEFORE-INDEX`
Beyond navigating directly to elements or sequences, Specter can navigate to specific *positions* or empty spaces between elements, allowing you to insert data before, between, or after existing values without overwriting them. The speaker explains: “Specter has an ability not just to navigate to a specific element or a specific sequence within a date structure but can actually navigate to a specific position and once you're at that position then if you do an insert then you can place things in between before or after which is really powerful.”
```mermaid flowchart LR accTitle: Positional Navigation for Insertions accDescr: Showing how BEFORE-ELEM, BEFORE-INDEX, and AFTER-ELEM position pointers for inserting items into a sequence. subgraph PointsVector ["Vector of Points"] P0["[0] Point 1"] P1["[1] Point 2"] end BE["BEFORE-ELEM"] -->|"Inserts at index 0"| P0 BI["BEFORE-INDEX 1"] -->|"Inserts between 0 and 1"| P0 BI -->|"Inserts between 0 and 1"| P1 AE["AFTER-ELEM"] -->|"Inserts at end"| P1 ```
The speaker experiments with various positional navigators: 1. **`AFTER-ELEM`**: Targeting `[:res FIRST:catalysts FIRST:points AFTER-ELEM]` and calling `setval` with `{:point "Z"}` appends the new point after the existing elements, expanding the collection from two points (`point-1`, `point-2`) to three (`point-1`, `point-2`, `point-Z`). 2. **`BEFORE-ELEM`**: Switching the navigator to `BEFORE-ELEM` inserts `point-Z` at the head of the vector. The speaker humorously apologizes for having nested the sample data so deeply: “Oh I did I put it so far nested down it takes us ages to find it each time sorry about that yeah that is expected now point Z is at the beginning of the sequence of this vector of maps not at the end.” 3. **`INDEXED-PATH` vs `BEFORE-INDEX`**: Wanting to insert an element in the middle, the speaker initially checks `nthpath` (or `INDEXED-PATH`), but realizes that navigates directly *to* the specified element index (replacing it like `nth`), rather than targeting the empty slot between indices. Checking Nathan Marz's documentation on GitHub and the Specter wikis for navigators and macros, the speaker finds `(BEFORE-INDEX 1)`. Running `setval` targeting `(BEFORE-INDEX 1)` successfully injects the new point cleanly between index `0` and index `1`.
---
[32:44](https://www.youtube.com/watch?v=rh5J4vacG98&t=1964s) Querying Arbitrary Tree Structures with `WALKER`
To conclude the demonstration, the speaker highlights Specter's built-in `WALKER` navigator, which recursively traverses an entire data structure without needing an explicit path schema. This is especially useful when data structures behave like arbitrary trees and you need to query or collect specific data types from leaf nodes.
Using `select` combined with `(WALKER number?) data`, Specter walks the entire tree and extracts every single number into a flat vector. Changing the predicate from `number?` to `string?` immediately pulls out all string values across every level of the structure. The speaker summarizes the utility: “Walker is a fantastic way to navigate to all of the nested leaves of any deeply nested data structure which tends to look like a tree and just make sure that you get all of them out and then you can apply a predicate to make sure you only see the ones that you care about.” The discussion then pivots toward parsing and transforming strings encountered along paths.
- `(select [(WALKER number?)] data)` extracts all numbers across arbitrary nesting depths. - `(select [(WALKER string?)] data)` pulls every string leaf across the structure. - The Specter GitHub repository provides comprehensive examples, video presentations by Nathan Marz, and detailed wiki pages covering navigators and macros.
[34:24](https://www.youtube.com/watch?v=rh5J4vacG98&t=2064s) Bi-Directional Transformations with `parser`, `parse`, and `unparse`
The host shifts focus to string manipulation inside nested data structures, taking on a specific challenge: rounding down all stop times to the nearest hour. If an entry has a stop time string like `"10.35"`, the goal is to transform it into `"10.00"`. Specter handles this kind of intra-value navigation through a bi-directional parsing mechanism that requires two companion functions: a `parse` function and an `unparse` function. Because Specter works by drilling down the navigational path to isolate target elements and then reconstructing the data structure on the way back up, it relies on this paired structure to keep transformations lossless and seamless.
As the host explains the underlying mechanics: “it as you navigate down and you do a pass it will call your pass function to split things up however you want and then you can continue manipulating it but then when it's reconstructing and doing the reverse it needs an unpassed function to put them back together”. Implementing this in Clojure requires splitting and joining strings:
```mermaid flowchart LR accTitle: Bi-directional Specter Parsing Flow accDescr: Navigation splits the target string via parse, modifies the value, and reconstructs the string via unparse. subgraph Downward ["Navigation Downwards"] A["Raw String<br/>'10.35'"] -->|"parse (str/split)"| B["Vector<br/>['10', '35']"] B -->|"LAST"| C["Minute String<br/>'35'"] end subgraph Upward ["Reconstruction Upwards"] C -->|"set-val '00'"| D["Modified Vector<br/>['10', '00']"] D -->|"unparse (str/join)"| E["Final String<br/>'10.00'"] end ```
- **The `parse` function**: Defined using `clojure.string/split`, taking the time string and splitting it on the literal dot character (`\\.`, escaping the regex dot wildcard). When passed `"10.35"`, it successfully evaluates to a two-element vector: `["10" "35"]`. - **The `unparse` function**: Defined using `clojure.string/join`, taking a dot `"."` separator and the collection `split-time` vector of strings. Calling `unparse` on `["10" "35"]` evaluates back to the joined string `"10.35"`, providing the exact inverse operation required by Specter.
[37:03](https://www.youtube.com/watch?v=rh5J4vacG98&t=2223s) Navigating and Rounding Down Values with `parser` and `LAST`
To apply this logic to the nested trip data structure, the host demonstrates how to assemble the Specter path using `set-val` (interchangeably tested with `select`). Navigating down past the points into each map, the path targets `:stop-time`. At that exact point, Specter provides the built-in navigator `parser`, which accepts the custom `parse` and `unparse` functions as arguments. By invoking `parser` inline, Specter translates each stop time string into its split vector representation on the fly during navigation.
The host emphasizes how useful `select` is during development: “you can always use select as you're building up your transforms or your cells to see where you've got to”. Inspecting the navigation with `select` confirms that the path lands squarely on vectors of split string components. From there, Specter's standard navigators can continue drilling deeper into the parsed data. Appending the navigator `LAST` targets specifically the minute position (`"35"`). Switching back from `select` to `set-val`, the host replaces all targeted minute values with `"00"`. Re-evaluating the full structure confirms that previous values like `"13.35"` and `"2.35"` are cleanly updated to `"13.00"` and `"2.00"`.
- `parser` bridges the boundary between nested data structures and string-level manipulation, allowing further Specter navigators (such as `LAST`) to operate directly inside the parsed vector. - The host concedes that rounding time strings like this is “a bit of a contrived example you might not do it in this way but it's just to show you how you can use other functions with your navigator to change how you're seeing the data and then you can manipulate it”. - The host highlights the power of Specter's automatic structural restoration: “and because of how Specter magically puts it all back together for you you get very powerful results with very little code”.
[40:02](https://www.youtube.com/watch?v=rh5J4vacG98&t=2402s) Conclusion and Enthusiasm for Specter
The video concludes with an endorsement of the library's utility across Clojure development. The host notes, “I'm very excited about Specter I use it all the time in all the projects that I write in Clojure”, having demonstrated why Specter is so compelling, how its two-phase navigation and reconstruction pipeline works, and how navigators like `parser`, `select`, and `set-val` simplify complex manipulations. Expressing the hope that viewers have caught the excitement and will explore the library in their own codebases, the host thanks the audience and signs off.
## Mentioned Specter, Clojure, Nathan Marz, Rich Hickey, JSON, Atom, proto-repl, GitHub
## Video description Learn how to elegantly manipulate nested data structures with Specter.
Specter: https://github.com/nathanmarz/specter
Language: http://clojure.org Editor: https://atom.io/ proto-repl: https://atom.io/packages/proto-repl proto-repl setup guide: https://gist.github.com/jasongilman/d... Atom syntax theme: https://atom.io/themes/borealis-syntax
[Source](https://www.youtube.com/watch?v=rh5J4vacG98)