2021 Miller User Survey results

[johnkerl]

There were about 40 responses to the 2021 Miller User Survey. It's deeply gratifying to get such thoughtful, useful feedback! I thought I'd share some of the findings.

Use-cases

These are all over the map, and pleasantly so. All along I intended Miller to be multi-purpose -- it seems many of us are solving the same problems, sometimes ad-hoc, and there was a people-empowering niche to fill. The results here confirm this. One of my favorite quotes ever was @eddelbuettel referring to Miller as an underappreciated swiss-army chainsaw.

File formats

Here too the results are all over the map -- but with a marked emphasis on CSV. When I originally wrote Miller, back in 2015, DKVP was the format I most often used (although I use primarily CSV, TSV, PPRINT, XTAB, and JSON now); DKVP was (and is) Miller's default file format. However, most people value CSV -- while each of Miller's supported formats is useful to some survey respondents, all respondents flagged CSV.

As a result of this feedback, the Miller 6 docs are more CSV-centric. Up-front examples that used the abixy sample DKVP file in the Miller-5 documents now use the example.csv file, and many examples now use CSV and JSON.

Documents

The most-common response was that the docs are just right. But a close second response was that it's too hard for people to find what they're looking for within that material.

Some free-form comments:

• There is a lot of material, but it's better to have too much than too little
• An ask for an acronyms dictionary
• More examples
• The docs give too much focus, too soon, on advanced material and too little focus on simpler (and more common) use-cases
• Most-important pages (such as the verb-list) were buried deep in, and should be a click or two away from the landing page

As a result of this feedback, the Miller 6 documents have been completely reworked relative to the Miller 5 documents.

• The site-generator has been ported from Sphinx to Mkdocs, with improved look and feel. As a bonus, the new Mkdocs structure is much more mobile-friendly, working well on desktop as well as mobile.
• The left-hand table of contents is always in view (a finger-tap away on mobile), showing current context within the doc site.
• Likewise, the in-page table of contents on the right-hand side allows for quick navigation for sections of the current page.
• The Search feature is at the top right of every page.
• Quick links is at the top center of every page, allowing repeat visitors quick access to most-used reference information.
• The new glossary is a particularly useful addition, thanks to an idea from @NikosAlexandris.
• Code snippets now have their command-line inputs on a darker background, with bold text -- thanks to an idea from @aaronwolen.
• The site structure is now multi-part: Getting started with extended narrative and introductory examples; Miller in more detail for next-level / early-on reading; FAQs and examples with non-theoretical deep-diving into specific use-cases; and lastly thorough reference sections for those times when people need the most precise information.
• The reference pages have been split up into more, but shorter, pages.
• In addition to the survey, more doc-related feedback is on GitHub Discussions at Feedback request for Miller 6 docs #621. @aborruso and @NikosAlexandris have contributed fantastic review and discussion.

Output colorization

In some recent covert art at https://github.com/johnkerl/miller I intended only to show visually that records, as lists of key-value pairs, are the common structure underlying Miller's handling of seemingly distinct file formats. This is, in fact, what makes it possible for Miller to be a swiss-army chainsaw. But one respondent was dismayed that this coloring was not a live feature.

I'd never thought of that, but realized it was not only doable but desirable -- and after all, jq does something similar with JSON files.

As a result, in Miller 6 we now have output colorization -- what was once cover art is now something we can use interactively.

Community

As a clear growth opportunity, most respondents were not aware of current Miller community options.

What should be next?

Development on Miller 6 continues unabated.

People were most interested in testing:

I hope to have a beta release of Miller 6 out soon for general evaluation.

As always, please keep the feedback coming! Six years ago, Miller was a personal project; good feedback is helping it to become much more than I had ever anticipated.

• https://github.com/johnkerl/miller/discussions
• https://github.com/johnkerl/miller/issues

[aborruso]

Hi @johnkerl thank you very much, really really interesting.

I have a stupid question: why is the sum of the percentages is more than 100 (in more than one graph)?

[eddelbuettel]

Maybe because the answers were non-exclusive?

[aborruso]

Maybe because the answers were non-exclusive?

probably you are right, thank you

[johnkerl]

@aborruso @eddelbuettel indeed! :)