rsh

rsh 0.74

rsh, or Rsh for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.74 of Rsh. This release includes improvements on handling signatures of exec and known externals, improved help, initial support for parse-time constants, new commands, and many improvements to our existing commands.

Where to get it

Rsh 0.74 is available as pre-built binariesopen in new window or from crates.ioopen in new window. If you have Rust installed you can install it using cargo install rsh.

NOTE: The optional dataframe functionality is available by cargo install rsh --features=dataframe.

As part of this release, we also publish a set of optional plugins you can install and use with Rsh. To install, use cargo install rsh_plugin_<plugin name>.

Themes of this release / New features

Known externals commands and exec now have "fall-through" signatures (merelymyself, WindSoilder, kubouchopen in new window)

A common pitfall in rsh when defining custom signatures using extern used to be that unrecognized arguments passed to the command would throw an error. Now, arguments are still checked against the extern signature but those that are not recognized are simply ignored. 

> extern `git checkout` []
> git checkout -b foo  # does not throw an error

exec uses similar style which fixes errors with tools like ssh and gdb that internally invoke exec. 

> rsh -c 'exec ls -l'  # does not throw an error

help is now more helpful (kubouchopen in new window)

For a long time, rsh had the option to provide custom help messages for commands via comments: 

# Some Command
#
# More description goes here
def some-command [] { 'foo' }

help some-command  # displays the comments in the help message

In this release, we allow user-defined help messages with aliases and modules. This goes hand-in-hand with a handful of new help subcommands to explore these help messages.

The help messages now also treat the first line followed by an empty line as a "brief" description displayed in summary tables generated by help commands, $rsh.scope.aliases, etc. The full description is available when querying a particular command/alias/module (e.g., help spam). This brief vs. full separation was already present for built-in commands, like path, but now it is applied also to all user-defined help messages.

An example of using custom module and alias help messages: Help messages for modules and aliases

The current batch of improvements can still be taken further. For example, custom help messages could possibly be defined also for variables and environment variables (via comments adjacent to let and let-env). We could also further improve the presentation of existing help xxx commands.

Initial support for parse-time constants (kubouchopen in new window)

This is a proof-of-concept that we plan to expand in the future.

Tip: We also addedd a new book sectionopen in new window with an in-depth explanation of rsh's parsing and evaluation, hopefully clearing up some confusion about things like "Why can't I source a dynamic path?". It also touches on the concept of parse-time constants.

A new const keyword is added to rsh to define "parse-time" constants. Constants defined with const behave the same as variables defined with let, but in addition, they are usable in some contexts that require values known at parse-time. Currently, this applies to files or names passed to use, overlay use, source, and source-env. For example, the following now works: 

# Source a file from a constant
> 'print hello!' | save --raw say_hello.rsh
> const fname = 'say_hello.rsh'
> source $fname
hello!

# Load an overlay from a constant name:
> module spam {
	export def foo [] { 'foo' }
}
> const name = 'spam'
> const new_name = 'eggs'
> overlay use $name as $new_name
> overlay list | last
eggs
> foo
foo

Only a limited subset of values is allowed to be a constant. In general, "simple" values, like strings or integers, and their collections (lists, records) are allowed but values requiring some evaluation (string interpolation, subexpressions, environment variables) are not allowed. The current selection is not set in stone, however, and might change in the future.

Some future direction we can take this:

  • Move parts of $rsh to be constant to allow things like source $rsh.config-path
  • Allow modules to have constants (module spam { const CONTENTS = [ 'eggs', 'bacon', 'sausage', 'spam' ] })
  • Some limited form of parse-time evaluation to allow static control flow, for example 
const fname = const if $rsh.os-info.name == 'windows' {
    'C:\Users\viking\spam.rsh'
} else {
    '/home/viking/spam.rsh'
}
overlay use $fname

In general, we want to be very conservative with parse-time constants and evaluation because it can be easy to introduce awkward side-effects and performance pitfalls. We plan to extend this only where it brings some tangible benefit to rsh's user experience.

New url encode command to percent-encode URLs (MehulGopen in new window)

To encode text that is used as a path component in a URL we now provide url encode.

By default it preserves the structure of a URL and only replaces invalid characters. With --all the whole string gets encoded. 

> 'https://example.com/foo bar' | url encode
https://example.com/foo%20bar
> 'https://example.com/foo bar' | url encode --all
https%3A%2F%2Fexample%2Ecom%2Ffoo%20bar

values command to programmatically interact with records (webbedspaceopen in new window)

This is a complement to columns, designed to allow the values of a record to be easily filtered and iterated over using the standard list tools like each and where. The name values is derived from similar functions in Ruby, Python and JavaScript. 

>  {a: "Happy", b: "new", c: "year"} | values
╭───┬───────╮
│ 0 │ Happy │
│ 1 │ new   │
│ 2 │ year  │
╰───┴───────╯

It can also operate on tables to convert them to lists-of-lists: 

>  [[a b]; [4 7] [5 8] [6 9]] | values
╭───┬───────────╮
│ 0 │ ╭───┬───╮ │
│   │ │ 0 │ 4 │ │
│   │ │ 1 │ 5 │ │
│   │ │ 2 │ 6 │ │
│   │ ╰───┴───╯ │
│ 1 │ ╭───┬───╮ │
│   │ │ 0 │ 7 │ │
│   │ │ 1 │ 8 │ │
│   │ │ 2 │ 9 │ │
│   │ ╰───┴───╯ │
╰───┴───────────╯

get, select, cell path access on tables will now error when encountering a hole (kubouchopen in new window, webbedspaceopen in new window)

Consider the following operations performed on a table:

  • [{foo: 'bar'}, {}] | select foo
  • [{foo: 'bar'}, {}] | get foo
  • [{foo: 'bar'}, {}].foo

Formerly, this would produce ['bar', null] - converting the table hole into a null. Now, however, they will produce an error. The original null-conversion behaviour can, as usual, be opted into using the -i flag for get and select: [{foo: 'bar'}, {}] | get -i foo produces ['bar', null]. (There are also plans for a future version of rsh to expand the cell path syntax to allow certain cell names to be "nullable" - converted to null if they don't exist.)

Behavior of -i/--ignore-errors flag for get and select when the entire column is absent has changed

Formerly, if select -i or get -i couldn't find any value for the given column, it would produce a single null: 

〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
null

This has been changed so that it now produces a table (or, in the case of get, a list) of all nulls: 

〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
[[foo]; [null], [null], [null]]

This change was made to make this flag work more consistently with default and compact: 

〉[{a:1} {b:2} {a:3}] | select -i a | default 0 a
[[a]; [1], [0], [3]]
〉[{a:1} {b:2} {a:3}] | select -i foo | default 0 foo
[[foo]; [0], [0], [0]]

As you can see, default in the above example can reliably populate an entire table column when some or all of the values don't exist.

Certain misused punctuation in def definitions are now errors (webbedspaceopen in new window, webbedspaceopen in new window)

The following misuses of punctuation in def definitions now produce errors:

  • Placing a comma between a long flag and its short alternative (e.g. def a [--foo, (-f)] {})
  • Consecutive commas, like def a [foo,,bar] {}
  • Consecutive colons, like def a [foo::int] {}
  • Using ^ in command names, like def ^a [] {}

$in now works in catch closures

$in in catch closures now behaves identically to how it does in other closures, like those given to each (it's equivalent to what would be the first named argument).

try { 'x' | math abs } catch { print $in } behaves the same as try { 'x' | math abs } catch {|e| print $e }.

MIME-types are supported in ls with an additional flag. (fdncredopen in new window)

To find out what application your operating system associates with a particular file, you can now use the --mime-type or -m flag on our ls command. This simplifies filtering for particular files and can help you dispatch files to particular programs. When opening files with rsh directly, open will still follow the same heuristics using file endings and the built-in from ... command parsers.

Regular expression queries are cached for performance (rgwoodopen in new window)

The primary motivation for this is to make regex and =~ operator uses in hooks and color_config closures more performant.

All built-in commands now declare their pipeline input and output types (sholderbachopen in new window)

A few releases back commands internally got the capability to declare not only the types of parameters but also pairs for the input and output on the pipeline. With this release we finally declare those input and output types for all core rsh commands. This can help you as user to see what a command expects from the pipeline and might return. We are exploring how rsh can leverage that for more useful diagnostics and completions. In the future we may introduce syntax for user-defined commands to declare their input and output types explicitly.

help for wrap command shows I/O type signatures

Breaking changes

Full changelog

rsh

Documentation

rsh_Scripts

reedline