rsh 0.85

rsh, or Rsh for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.85 of Rsh. This release adds the first uutils command, unlocks more constant evaluation at parse time, and polishes many commands.

Where to get it

Rsh 0.85 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install rsh.

NOTE: The optional dataframe functionality is available by cargo install rsh --features=dataframe.

As part of this release, we also publish a set of optional plugins you can install and use with Rsh. To install, use cargo install rsh_plugin_<plugin name>.

Themes of this release / New features

rsh + Uutils = ❤️

📢The rsh team is thrilled to announce 🥳 that we've begun working with the uutils/coreutils team on integrating some of their foundational core utilies into rsh. With this release, we've started with the coreutils cp command, which we've temporarily named ucp. We're starting with ucp to allow broad testing while the current rsh cp command remains the default. We've already found one bug in the coreutils cp command and you might find others. Once it stabelizes, probably with the 0.86.0 release, we'll remove the rsh cp command and rename ucp to cp. In keeping with the rsh style, we've only added a handful of parameters. We can, and probably will, add more if the community determines we need them. We're so very excited and would like to thank terts, from the coreutils team, for his excellent help and special appreciation to dmatos2012 for the, very iterative, first uutils/coreutils integration PR #10097 for the cp command. To read more about how this came to be, checkout our blog post.

Quite a few bug fixes

Thanks to all the contributors below for helping us solve issues and bugs 🙏

author	description	url
@rgwood	Fix watch not detecting modifications on Windows	#10109
@ito-hiroki	Fix tab completion order of directories to consistent with order of files	#10102
@herobs	Fix 9156 endian consistency	#9873
@ayax79	fixed usages of deprecated chrono DateTime::from_utc	#10161
@zhiburt	Fix #10154	#10162
@ofek	Fix example history command pipeline	#10220
@dead10ck	Fix unit tests on Android	#10224
@amtoine	fix default after an empty where	#10240
@IanManske	Fix rm on macOS	#10282
@horasal	handle empty pipeline while parsing let (fix Issue10083)	#10116
@dmatos2012	Fix variables not allowed in ucp	#10304
@sholderbach	Update crates-ci/typos and fix new typos	#10313
@GomesGoncalo	fix #10319: allow json request of value type list	#10356
@jntrnr	fix 'let' to properly redirect	#10360
@amtoine	fix the pretty printing of failing tests in std	#10373
@J-Kappes	fix input --until-bytes: now stops at any of given bytes	#10235
@zhiburt	rsh-table: Fix expand table unnessary color in trail head config when wrap is used	#10367
@fdncred	fix some new chrono warnings	#10384
@horasal	prevent crash when use redirection with let/mut	#10139
@horasal	Allow operator in constants	#10212
@zhiburt	rsh-table: Patch restore lead trail space bg color	#10351
@zhiburt	rsh-table: Strip custom color in the header when used on border	#10357

Consistent use of float for our floating point type

Breaking change

See a full overview of the breaking changes

rsh currently supports two types to represent numbers without units: int for integer numbers and float for floating point numbers. The latter type was in important places incorrectly referred to as decimal. This hid the fact that as floating point numbers they have limited precision in some circumstances but higher performance compared to a decimal-encoded number.

With this release we fix this inaccuracy.

This means we introduce the commands into float and random float and deprecate the commands into decimal and random decimal. The old commands will be removed with the next release but continue to work for this release with a warning. Please use into float instead of into decimal and random float instead of random decimal.

After the type returned by describe has been float for a while, we now also change which type name is allowed when specifying command argument or input/output types.

# Argument with a specfic type
def foo [bar: float] {}
# Command taking only floating point input from the pipeline and returning the same type.
def baz [] float->float {}

Previously both float and decimal were supported in those positions.

Some updates on explore

Breaking change

See a full overview of the breaking changes

The explore built-in commands have changed a bit during this release.

Now, it supports Vim bindings out of the box thanks to @amtoine in #9966. @rgwood has simplified the available configuration entries which used to make the command less easy to understand in #10258#10259 and #10270.

Thanks to him again, one can now quit the explorer by hitting any of ctrl+c, ctrl+d or ctrl+q (see #10257).

Improvements to parse-time evaluation

Conditional source and use is now possible.

The rsh core team has been asked quite a lot of times to be able to do parse-time conditional sourceing or useing in the configuration, e.g. to load different things depending on the system:

if $rsh.os-info.name == "windows" {
    source "my_windows_config.rsh"
} else {
    source "my_unix_config.rsh"
}

The above snippet does not work because rsh's scoping rules keep the sourced values inside the block and the sourcing would not have any visible effect outside of the if/else statement (except environment changes).

While scoping still works the same, a series of PRs by @kubouch made conditional source/use/overlay use possible:

• #9499, allows parse-time evaluation of commands, pipelines and subexpressions (see also Looking for help!)
• with #10326, if can now be evaluated at parse time
• finally, #10160 makes the $rsh built-in variable a true constant 🥳

What does this all mean? One can now write something very similar to the proposed snippet above! For example:

const WINDOWS_CONFIG = "my_windows_config.rsh"
const UNIX_CONFIG = "my_unix_config.rsh"

const ACTUAL_CONFIG = if $rsh.os-info.name == "windows" {
    $WINDOWS_CONFIG
} else {
    $UNIX_CONFIG
}

source $ACTUAL_CONFIG

Looking for help!

#9499 allows running selected commands at parse time. For example,

const f = ($rsh.default-config-dir | path dirname)

is possible because path dirname was manually ported to allow parse time evaluation. Only a very limited subset of rsh's commands is currently allowed to do so:

• str length
• path commands
• Some core commands: describe, ignore, version, if

We would like to expand this set to allow more commands to run at parse time. If you'd like to help us porting more commands, we'd welcome your help! Please, see #10239 for more information. Porting the commands is not difficult, but requires a bit of consideration, so it's better to reply to the issue or reach out to the core team beforehand.

Improving accessibility

Up until now, an issue for users using a screen reader was that rsh errors and tables are quite fancy with some unicode characters. The issue is that screen reader might have trouble reading them, making the understanding of what's going on the REPL very hard...

For tables, it's quite easy, you can set $env.config.table.mode to something like "basic" or "none" and you should be good! But errors remained fancy

Error: rsh::shell::external_command

  × External command failed
   ╭─[entry #4:1:1]
 1 │ foo
   · ─┬─
   ·  ╰── did you mean 'for'?
   ╰────
  help: No such file or directory (os error 2)

@JoaquinTrinanes did implement screen reader-friendly errors in #10122 which will hopefully make the experience a lot better when it comes to errors.

More support for more platforms

In this release, @dead10ck made it possible to use rsh in Termux (#10013) and fixed a bug on Android (#10225).

Improved history isolation

Breaking change

See a full overview of the breaking changes

Like many other shells, history between open sessions can be isolated in rsh. However, the former implementation had the disadvantage that it also blocked access to history of former sessions. It also didn't isolate hints. This is now fixed with !10402 by @Hofer-Julian. Since this was merged shortly before the release, this feature isn't the default yet. The file_format has to be changed to "sqlite" and isolation has to be set to true. You can find those in the config file under $rsh.config-path. Please test this so we can fix problems before history isolation becomes the default.

Enhancing the documentation

Thanks to all the contributors below for helping us making the documentation of rsh commands better 🙏

author	description	url
@Hofer-Julian	Add notice to enable develop mode on Windows	#10111
@fdncred	update query web example because wikipedia changed their page	#10173
@alsuren	Point from keybindings help to the book's reedline chapter	#10193
@sholderbach	Document that open looks up from subcommands	#10255
@balupton	readme: add dorothy to supported by	#10262
@brunerm99	feat: Search terms for use, while, and range (#5093)	#10265
@amtoine	add case-insensitive example to where	#10299

Help with tests

Some more technical work but very helpful to make the source code of rsh better, so thanks to our contributors who did improve the tests, often going through the whole source base and doing tideous find and replace 🙏

author	description	url
@J-Kappes	Tests: clean up unnecessary use of pipeline()	#10170
@sholderbach	Remove dead tests depending on inc	#10179
@sholderbach	Simplify rawstrings in tests	#10180

Changes to commands

As usual, new release rhyms with changes to commands!

• a bunch of new columns have been added to the ps commands:
  • @fdncred in #10275 and #10344
  • @WindSoilder in #10347
• @nanoqsh has made the behaviour of append and prepend consistent regarding ranges in #10231
• @Tiggax has started working on making select, get and reject more consistent in #10163 and #10216
• Breaking change: math constants have been moved from built-in commands to true constants in the standard library by @amtoine in #9678
• the into duration command now accepts integers as arguments and the unit can be specified via an option with the changes introduced by @amtoine in #10286
• thanks to @fdncred in #10354, the commandline command can now move you to the end of the line, no need to "move 1000 characters to the right" anymore

Some table themes

Want more delight? @fdncred got you covered. He added a bunch of new table themes / modes in #10279.

echo is evolving

Coming from other shells, e.g. POSIX ones like Bash or Zsh, the distinction between rsh's echo and print commands and the behaviour of echo itself could be confusing 🤔

• print is a more programming-language-friendly command: it does print its arguments to the terminal directly, consuming them and thus not allowing to pipe the "output" of print further.
• echo was only used to create values and pass them down a pipeline, e.g. echo "foo" | str length

@jntrnr made the behaviour of echo a bit more general

• it will print to the terminal if not redirected
• it will pass the value down to the pipeline if redirected

echo "foo"  # will behave exactly as `print` does
echo "foo" | str length  # will compute the length of `"foo"` and forward the result without
                         # "printing" it unless it's the last command being run

Pythonesque operators removal

Breaking change

See a full overview of the breaking changes

Coming from Python, things like

3 * "foo"
[1, 2] * 10

would probably appear familiar.

However, they could lead to some strange internal behaviours and hard to debug issues 😕

> [3, "bob", 4] | reduce --fold 1 {|x, y| $x * $y}
bobbobbobbobbobbobbobbobbobbobbobbob

Note in the example above, we are mixing integer and string multiplication, which might get weird!

In this release, we decided to remove the string and list scalar multiplication in #10292 and #10293 from @sholderbach.

However, we do not want to leave you without any other way to achieve the same, this is why @amtoine did implement the repeat command in the standard library in #10339

• bring it into your scope with use std repeat
• you can do scalar string multiplication with something like "foo" | repeat 3 | str join
• you can do scalar list multiplication with something like [1, 2] | repeat 3 | flatten

Optimizations

#10378 Switched the default allocator to mimalloc as it's shown to reduce startup time by upwards of 30% on Windows. If it does not build on unique platforms rsh can be built without this feature, then using the platform's default allocator.

New commands

• into float as a replacement for into decimal
• random float as a replacement for random decimal

In the standard library we added the following commands:

• std repeat as a command to repeat an element n times in a list.
• std formats from ndjson as a way to directly open newline-delimited JSON recrods. You can use open for files with the .ndjson ending if you use std formats "from ndjson".
• std formats from jsonl as a way to directly open newline-delimited JSON records. You can use open for files with the .jsonl ending if you use std formats "from jsonl".

Deprecations

Deprecated commands

• into decimal: Use into float instead
• random decimal: Use random float instead

Breaking changes

• Plugin authors need to update plugins after the span refactor
• Updated plugins need to be recompiled

• #10235 input --until-bytes now stops at any of given bytes
• #10333 Consistently use float instead of decimal in type specifications
• #10293 Remove pythonic string * list multiplication
• #10292 Remove pythonic int * list multiplication
• #10338 Change echo to print when not redirected
• #10259 explore: remove :config, :show-config, :tweak commands
• #9929 Rename the types with spaces in them to use -
• #10254 Change LOG_FORMAT to rsh_LOG_FORMAT in rsh-std library
• #9678 Move math constants to standard library
• #10231 Make append/prepend consistent for ranges
• #9966 Add support for Vim motions in explore
• #10042 Spanned Value step 1: span all value cases
• #10103 Create Record type
• #10064 Fully remove str replace --string option after deprecation. Just use str replace instead.

Full changelog

rsh

• tokatoka created
  • Add 2 fuzzers for rsh-path, rsh-parser
• sholderbach created
  • Clippy in tests
  • Update internal use of decimal to float
  • Optimize use of range in std repeat
  • Remove python-like string multiplication
  • Invert &Options to Option<&T>
  • Remove pythonic int * list behavior
  • Rename random decimal to random float
  • Rename into decimal to into float
  • Deref &String arguments to &str where appropriate
  • Use slices directly instead of &Vec
  • Bump calamine and fix clippy
  • Update crates-ci/typos and fix new typos
  • Remove Cargo.lock in subcrates
  • Move spellcheck config into .github folder
  • Remove codecov.io setup
  • Document that open looks up from subcommands
  • Tweak contributor image to include more users
  • Update crossterm/ratatui/dev-reedline
  • Simplify rawstrings in tests
  • Remove dead tests depending on inc
  • Keep arrow2 out of basic --workspace build
• fdncred created
  • fix some new chrono warnings
  • add helper switch to move cursor to end of buffer
  • add a few more columns to linux ps -l output
  • silence some ucp warnings
  • allow update to use metadata
  • add 6 more table themes
  • respect a users locale with the right prompt
  • add more ps columns in Windows
  • update format date when using %x %X %r
  • Restore rsh_LIB_DIRS and rsh_PLUGIN_DIRS defaults
  • allow --login to be used with rsh's --commands parameter
  • add plugin path when there are no signatures
  • update query web example because wikipedia changed their page
  • name hooks internally
  • bump rust-toolchain to 1.70.0
• FilipAndersson245 created
  • Adds mimalloc as default feature.
• zhiburt created
  • rsh-table: Fix expand table unnessary color in trail head config when wrap is used
  • rsh-table: Strip custom color in the header when used on border
  • rsh-table: Patch restore lead trail space bg color
  • rsh-explore: Refactorings
  • Fix #10154
• J-Kappes created
  • fix input --until-bytes: now stops at any of given bytes
  • Tests: clean up unnecessary use of pipeline()
• amtoine created
  • fix the pretty printing of failing tests in std
  • add std repeat command to replace "foo" * 3
  • add case-insensitive example to where
  • allow into duration to take an integer amount of ns
  • rename the types with spaces in them to use -
  • fix default after an empty where
  • move math constants to standard library
  • support tab completion cycling
  • add support for Vim motions in explore
• WindSoilder created
  • Ps: add cwd column on linux and macos
  • return error when user break sleep by ctrl-c
• jntrnr created
  • fix 'let' to properly redirect
  • Change echo to print when not redirected
  • remove profiling from rsh's hot loop
  • Move Value to helpers, separate span call
  • Spanned Value step 1: span all value cases
• GomesGoncalo created
  • fix #10319: allow json request of value type list
• Tiggax created
  • Remove select error if same row/column is provided
  • update reject to be able to recive arg list
  • reject multiple row args support
• kubouch created
  • Allow parse-time evaluation of if
  • Remove leftover const eval file
  • Add rsh_VERSION environment variable on startup
  • Make $rsh constant
  • Allow parse-time evaluation of calls, pipelines and subexpressions
• hustcer created
  • Update Rsh to v0.84 for release and nightly-build
  • Upgrade softprops/action-gh-release to v0.1.15 for release and nightly build workflow
• geniusisme created
  • provide env to commands and try to start provided path
• utouto97 created
  • add 'from ndjson' into standard library
  • refactor input command
• nanoqsh created
  • Keep order for par-each
  • Make append/prepend consistent for ranges
• app/dependabot created
  • Bump bytesize from 1.2.0 to 1.3.0
  • Bump tempfile from 3.7.0 to 3.8.0
  • Bump actions/checkout from 3 to 4
  • Bump itertools from 0.10.5 to 0.11.0
  • Bump rust-embed from 6.8.1 to 8.0.0
  • Bump git2 from 0.17.2 to 0.18.0
  • Bump winreg from 0.50.0 to 0.51.0
  • Bump notify-debouncer-full from 0.2.0 to 0.3.1
• dmatos2012 created
  • Fix variables not allowed in ucp
  • use uutils/coreutils cp command in place of rsh's cp command
• dzorya created
  • Added a comment that perl is required for feature static-link-openssl…
• nibon7 created
  • Make cursor_shape optional
  • Exit early when encountering parsing errors
  • Auto format let-else block
  • Use built-in is_terminal instead of is_terminal::is_terminal
  • Don't use oldtime feature of chrono
• alsuren created
  • signpost 'input list --types [key]' from 'keybindings list'
  • Point from keybindings help to the book's reedline chapter
• IanManske created
  • Fix rm on macOS
  • Restore initial foreground process group on exit
  • Create Record type
• rgwood created
  • Start removing colour config from explore
  • Exit explore on ctrl+c/d/q
  • explore: remove :config, :show-config, :tweak commands
  • Remove exit_esc and show_banner config from explore
  • Fix watch not detecting modifications on Windows
• brunerm99 created
  • feat: Search terms for use, while, and range (#5093)
• balupton created
  • readme: add dorothy to supported by
• stormasm created
  • change LOG_FORMAT to rsh_LOG_FORMAT in rsh-std library
  • Move hook to rsh_cmd_base
  • remove warnings in rsh_command tests
  • update rust-toolchain doc to 4 weeks from 3
• horasal created
  • Allow operator in constants
  • treat path contains '?' as pattern
  • prevent crash when use redirection with let/mut
  • skip comments and eols while parsing pipeline
  • handle empty pipeline while parsing let (fix Issue10083)
  • Add encoding auto-detection for decode
• dead10ck created
  • Clean up trash support on Android
  • Fix unit tests on Android
  • upgrade nix to 0.27
  • Support Termux
• ofek created
  • Fix example history command pipeline
• MasterMach50 created
  • changed default env file to use $rsh.home_path to find home
• matthias-Q created
  • feat: allow from csv to accept 4 byte unicode separator chars
  • Allow for .parq file ending as alternative to .parquet
• ayax79 created
  • fixed usages of deprecated chrono DateTime::from_utc
  • Updating polars and sqlparser versions
• SED4906 created
  • Update removed "MDI" icons to current MD icons
• JoaquinTrinanes created
  • Screen reader-friendly errors
• Hofer-Julian created
  • toolkit: Renames pretty-print-command
  • Add notice to enable develop mode on Windows
• herobs created
  • Fix 9156 endian consistency
• ito-hiroki created
  • Fix tab completion order of directories to consistent with order of files

Extension

• nerditation created
  • use which to search rsh location, add icon too.
• adamcstephens created
  • fix shebang/firstLine detection

Documentation

• Gryff created
  • Change direnv example to upload $env.PATH as a list
• edhowland created
  • Changed Python plugin example to be more portable
• dnsem created
  • Support for standard input in script with shebang
• petrisch created
  • DE translation for custom_completions.md
  • Typos in custom_completions
• mb21 created
  • Cookbook setup: fix command to append to PATH
• connorjs created
  • Update variables_and_subexpressions.md - remove obsolete section
  • Update working_with_lists.md - format date
• JoaquinTrinanes created
  • Fix external completers typo
  • Simplify external completer
  • Error style section
• hustcer created
  • Update min required node version and some node modules
• sholderbach created
  • Document that open looks up froms in scope
• stormasm created
  • add a link to the contributor book at the end of the plugins chapter
• alsuren created
  • Point out where command docs should be edited
• dclausen created
  • Update cheat_sheet.md
• simonboots created
  • Fix small typo in modules.md
• leetemil created
  • Update docs for exiting shell in shells
• adamchalmers created
  • Clarify how to check config file paths
  • Link to escape when discussing escapes
• follower created
  • Move launch instructions earlier in page.
• jamesarch created
  • fix missing command
• narve created
  • Update loading_data.md, correct link to http command
• dlamei created
  • fix typo in book
• lomm28 created
  • rsh cheat sheet added
• fdncred created
  • update fdncred's item + typo
• rgwood created
  • Update Reilly's comments in birthday blog post
• app/github-actions created
  • Compressed Images
• jntrnr created
  • Add rsh birthday post

rsh_Scripts

• amtoine created
  • fix scripts
  • add a typeof command
  • add $.type to package.nuon
  • add the random-bytes benchmark
• WindSoilder created
  • Improve background job.rsh
  • use closure instead of block
• Zinvoke created
  • FEATURE: add catppuccin-mocha theme
• brunerm99 created
  • Change rsh_conda list to output active status of environment as well (#604)
• icp1994 created
  • feat(hook): add rusty-paths.rsh
• selfagency created
  • Add support for fnm
• fnuttens created
  • Replace exa by eza aliases
• maxim-uvarov created
  • Update bar function
• StripedMonkey created
  • Expand git completions
• AntoineSebert created
  • Add dfr before into df and into rsh

Reedline

• sholderbach created
  • Remove old actions-rs/cargo
  • Add a configuration to codecov.io
  • Setup coverage with codecov.io
  • Bump crossterm to 0.27.0
• ysthakur created
  • Let prompts choose to repaint on enter
• Hofer-Julian created
  • Minor improvements
  • Fix read_line docs
• Abdillah created
  • Add Kitty protocol keyboard enhancement support