rsh 0.84.0

rsh, or Rsh for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.84.0 of Rsh. This release adds exporting constants from modules, scope commands improvements, cosmetic changes, and many smaller changes to our commands.

Where to get it

Rsh 0.84.0 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install rsh.

NOTE: The optional dataframe functionality is available by cargo install rsh --features=dataframe.

As part of this release, we also publish a set of optional plugins you can install and use with Rsh. To install, use cargo install rsh_plugin_<plugin name>.

Themes of this release / New features

(Major Breaking Change!) str replace matches substring by default (kubouch)

Breaking change

See a full overview of the breaking changes

str replace now matches a substring instead of a regular expression by default and the -s/--string flag is deprecated (will be removed altogether in 0.85). Matching a substring by default makes it more consistent with the rest of the rsh's commands.

💡 Note! Since str replace is a widely used command, many scripts are going to break. Fixing it is easy:

• If you used str replace without the -s/--string flag, you used regex matching and you should add --regex flag to keep the previous functionality.
• If you used str replace with the -s/--string flag, you will get a deprecation warning. To get rid of the warning, simply remove the flag.

Constants from modules (WindSoilder, kubouch)

You can now use constants in modules and export them. Calling use on a module will create record containing all module's constants. You can also import the constants directly, for example with use module.rsh const-name, without creating the module's record. Example:

> module spam {
    export const X = 'x'
    export module eggs {
        export const Y = 'y'
        export const viking = 'eats'
    }
}

> use spam

> $spam
╭──────┬───────────────────╮
│ eggs │ {record 2 fields} │
│ X    │ x                 │
╰──────┴───────────────────╯

> $spam.eggs
╭────────┬──────╮
│ Y      │ y    │
│ viking │ eats │
╰────────┴──────╯

> use spam eggs viking

> $viking
eats

The exported values are true constants which means that you can use them in some scenarios where you cannot use normal variables, such as:

> module files {
    export const MY_CONFIG = '/home/viking/.config/rsh/my_config.rsh'
}

> use files MY_CONFIG

> source $MY_CONFIG  # sources '/home/viking/.config/rsh/my_config.rsh'

New options for more compact table display (@zhiburt, #9796)

To save screen space, you can now display tables with the header directly displayed on the border through the $env.config.table.header_on_separator option.

Additionally you can now configure the padding left and right of the table cell content either separately or all at once to adjust the layout for your readability needs.

More consistent format ... commands (@sholderbach, @WindSoilder)

Breaking change

See a full overview of the breaking changes

With this release we moved several commands that serve to produce string output from a particular data type as format subcommands.

1. date format has been renamed to format date (#9788).
2. into duration --convert was previously used to provide string representations of durations. This option has been removed and you should now use format duration. (#9902)
3. format filesize that was previously moved into the --features extra set is back in the core (#9978).

scope commands enhancements (kubouch, kubouch)

scope commands received a significant amount of improvements making them more useful for advanced introspection of rsh's definitions.

• scope externs now lists known externals (defined with extern)
• scope aliases is now fixed and lists names and usages of the aliases, not of the aliased commands (this used to cause some confusion)
• scope modules now lists all the exportable of the module, including submodules with their contents
• all scope commands now list the definition's ID which can be used for reliable tracking of definitions. For example, a command foo imported from a module spam using use spam will be named spam foo, therefore, relying solely on names of the definitions can sometimes be misleading.
• module_name field is no longer present
• scope variables now denotes whether a variable is a constant or not

http commands now handle headers better (@jflics6460, @ineu

You can now pass headers directly as a record to the -H flag.

http get -H {cookie: "acc=foobar"} http://example.com

When running a http command with --full to get additional metadata about the response you can now get both the request and the reponse header

http get --full -H {cooke: "my=cookie"} | get headers | get response

Work on the optional dataframe features (@ayax79)

@ayax79 has been working hard to update our dataframe command to work again with the stricter type check by consolidating internal logic (#9860, #9951).

Furthermore, dfr open now suppports the avro exchange format and a new dfr to-avro command was added in #10019

Changes to other commands

Since last release, some commands have changed, here is a list of some interesting changed

• @atahabaki in #9841, #9856 and #9940: Some refinement of the str expand command (it's a really cool command, i recommend you check it out 😃)
• @fdncred in #9987: Allow select to take a variable with a list of columns
• @fdncred in #10048: Allow int as a cellpath for select

Command organization (@storm, @jntrnr)

In our efforts towards stabilization we moved several commands either behind the --features extra build-flag or back into the core set of commands. The special str *-case commands for example have been moved out of the core set, while the bytes commands to deal with bytes in arbitrary encodings have returned to the core set. Furthermore all commands in the core set should now have helpful categories associated with them.

Deprecation and removal of commands (@amtoine, @sholderbach)

We previously used the term "deprecation" pretty loosely to describe the removal of commands while emitting a helpful error. We intend to now start to use deprecation to warn you before the final removal of a particular option or command. When we removed a command we will now properly refer to it as removal and started adding warnings where appropriate (e.g. str replace --string). As part of this update we removed some very old error helpers nudging you to upgrade to new commands.

Documentation

Thanks to @rgwood, @sholderbach, @kubouch and @fdncred the documentation has become a bit better in #9961, #9996, #10004 and #10057.

Notable bugfixes

• A panic when parsing the context of let assignments has been fixed (@mengsuenyan in #9851)
• Invoking --help on a script with a def main command will now report the script's name instead of a more confusing main (@jntrnr in #9948)
• Serious bugs in parse that produced incorrect output when given a long input stream have been fixed (@panicbit in #9925, #9950)

Bugfixes

Thanks to all the contributors who tackled one or more bugs!

Name	Link	Description
@mengsuenyan	#9853	Fix ~ | path type returning empty string
@mengsuenyan	#9851	Fix the panic when type a statement similar to let f = 'f' $ in the rsh
@jntrnr	#9893	Revert #9693 to prevent CPU hangs
@NotLebedev	#9935	Nothing has the correct return type
@amtoine	#9947	Force version to update when installing with toolkit.rsh
@amtoine	#9967	Fix panic with lines on an error
@rgwood	#9990	Fix watch not handling all file changes
@nibon7	#9784	Fix a crash when moving the cursor after accepting a suggestion from the help menu
@meskill	#10007	Fix parser to not update plugin.rsh file on rsh startup
@zhiburt	#10011	rsh-table: Fix padding 0 width issues
@3lvir4	#10012	Remove potential panic from path join
@kubouch	#10046	Fix wrong path expansion in save
@zhiburt	#10050	rsh-table: Fix issue with truncation and text border
@jntrnr	#10052	Fix default_env.rsh after latest changes to str replace
@fdncred	#10067	Allow return to return any rsh value
@fdncred	#10063	Fix into datetime to accept more date/time formats

And also to those who did tackle purely technical challenges!!

Name	Link	Description
@IanManske	#9909	Enable macOS foreground process handling
@jntrnr	#9933	Add tests for script subcommands
@jntrnr	#9936	Fix a couple clippy warnings
@IanManske	#9927	Make Value::columns return slice instead of cloned Vec
@jntrnr	#9949	Move help commands to use more structure in signatures
@fdncred	#9958	Update strip-ansi-escapes to use new api
@rgwood	#9971	Put heavy dataframe dependencies behind feature flag
@sholderbach	#9974	Fixup dataframe build after #9971
@meskill	#9976	test: Clear parent environment to prevent leakage to tests
@kubouch	#10036	Add additional span to IncorrectValue error

Breaking changes

• #9902 date format has been renamed to format date for consistency.
• #9788 The option into duration --convert to format durations has been removed. Use the format duration command instead.
• #10038 str replace now by default matches to strings by default. If you want to use regexes you need to use the -r flag. The --strings flag has been removed
• #9632 To be unambiguous the duration type now does not report months or years. The largest unit reported is weeks as they have a consistent length compared to months or years.
• #9926 We moved some of the more exotic str case-commands behind the --features extra build-flag. This affects:
  • str camel-case
  • str kebab-case
  • str pascal-case
  • str screaming-snake-case
  • str snake-case
  • str title-case
• #10053 make the charpage optional for std clip
• #10023 The output of the scope commands has changed

Internal breaking change only affecting plugin authors

• #9927 Make Value::columns return slice instead of cloned Vec

Full changelog

rsh

• amtoine created
  • remove Clippy flags from the PR template
  • Revert "deprecate --format and --list in into datetime (#10017)"
  • make the charpage optional for std clip
  • deprecate --format and --list in into datetime
  • fix the signature of input list
  • fix the default config for explore
  • fix panic with lines on an error
  • add a test to make sure "nothing" shows up as "nothing" in help
  • force version to update when installing with toolkit
  • remove old deprecated commands
  • Fix default prompt indicators
• fdncred created
  • update install/build scripts to include --locked
  • allow help to return a Type::Table
  • try and fix into datetime to accept more dt formats
  • allow return to return any rsh value
  • enable/update some example tests so they work again
  • try to document the more obscure testbin commands
  • allow int as a cellpath for select
  • allow select to take a $variable with a list of columns
  • update strip-ansi-escapes to 0.2.0 and the latest reedline
  • update strip-ansi-escapes to use new api
  • add keybinding for search-history
  • add header_on_separator options to default_config.rsh
  • updates let-env signature to remove required params
  • remove vectorize_over_list from python plugin
  • update format signature to allow record to be passed in
  • update items signature to allow any output
  • update char signature with Table
  • update to current reedline
  • bump to dev version 0.83.2
• jntrnr created
  • Revert "pin serde to avoid https://github.com/serde-rs/serde/issues/2538"
  • move 'bytes' back to commands
  • pin serde to avoid https://github.com/serde-rs/serde/issues/2538
  • fix default-env after latest changes
  • Rename main to script name when running scripts
  • Auto-expand table based on terminal width
  • fix the Cargo.lock file
  • Move help commands to use more structure in signatures
  • Fix a couple clippy warnings
  • Add tests for script subcommands
  • Revert "Add an option to move header on borders"
  • Re-align how prompt indicators work
  • Simplify default style and match Rust code to config
  • Revert 9693 to prevent CPU hangs
• sholderbach created
  • Polish CONTRIBUTING, add Rust style
  • Remove global clippy -A from toolkit.rsh
  • Remove clippy global -A from CI
  • Pin reedline to 0.23.0
  • Rename misused "deprecation" to removal
  • Improve I/O types of into decimal(/float)
  • Add search terms to reject
  • Move format duration/filesize back into core
  • Fixup dataframe build after #9971
  • Add format duration to replace into duration --convert
  • Update unicode-linebreak to 0.1.5
• kubouch created
  • Remove "let config" warning
  • Simplify virtualenv testing
  • Recursively export constants from modules
  • Add a few more fields to scope commands
  • Add additional span to IncorrectValue error
  • Allow exporting extern-wrapped
  • Fix wrong path expansion in save
  • Change str replace to match substring by default
  • Remove dead code from tests
  • Sort entries in scope commands; Fix usage of externs
  • Refactor scope commands
  • Fix example for extern-wrapped
• zhiburt created
  • rsh-table: fix issue with truncation and text border
  • rsh-table: Fix padding 0 width issues
  • rsh-table/ Add table.padding configuration
  • Add an option to set header on border (style)
  • Add an option to move header on borders
• sitiom created
  • Change Winget Releaser job to ubuntu-latest
• ineu created
  • Make http -f display the request headers. Closes #9912
• 3lvir4 created
  • Remove potential panic from path join
• ayax79 created
  • Expose polars avro support
  • rsh table list columns -> dataframe list columns. Explode / Flatten dataframe support.
  • Merged overloaded commands
• app/dependabot created
  • Bump quick-xml from 0.29.0 to 0.30.0
  • Bump rstest from 0.17.0 to 0.18.1
• meskill created
  • fix(rsh-parser): do not update plugin.rsh file on rsh startup
  • test: clear parent envs to prevent leakage to tests
• nibon7 created
  • Fix a crash when moving the cursor after accepting a suggestion from the help menu
• rgwood created
  • Fix watch not handling all file changes
  • Put heavy dataframe dependencies behind feature flag
  • Fix cross-compiling with cross-rs
  • Fix match example whitespace
• panicbit created
  • parse: collect external stream chunks before matching
  • do not emit None mid-stream during parse
• bobhy created
  • Fix duration type to not report months or years
• IanManske created
  • Make Value::columns return slice instead of cloned Vec
  • Enable macOS foreground process handling
  • Replace &Span with Span since Span is Copy
• atahabaki created
  • str-expand: update bracoxide to v0.1.2, fixes #9913
  • str-expand: add path flag
  • str-expand: Add Escaping Example
• stormasm created
  • Categorification: move commands histogram and version out of the default category
  • Categorification: move from Default category to Filters
  • Categorification: move Path commands out of the default category
  • Categorification: graduate nuon --- from the experimental category to the formats category
  • Categorification: move uncategorized String commands to Category::Strings
  • Cratification: move some str case commands to rsh-cmd-extra
• NotLebedev created
  • Nothing return type
• WindSoilder created
  • rename from date format to format date
  • Module: support defining const and use const variables inside of function
• mengsuenyan created
  • Fixed the panic when type a statement similar to let f = 'f' $ in the rsh
  • fixed the bug ~ | path type return empty string
• app/ created
  • Turn bare URLs into cliclable links
• jflics6460 created
  • Accept records for http subcommand headers (-H)

Extension

• balupton created
  • readmde: close #148 - link extension page

Documentation

• jwarlander created
  • Use '--locked' in cargo install snippet for dataframe feature
• hustcer created
  • Update table mode config doc, fix #1007
  • Finish let-env removal in Chinese translation
  • Upgrade some dependencies, and fix some broken assets import
• conqp created
  • Update command to list aliases
• sholderbach created
  • Finish let-env removal in German translation
• amtoine created
  • remove last mentions to let-env
  • patch: release notes for 0.83.1
• BrewingWeasel created
  • Remove unused empty column
• rgwood created
  • Cookbook cleanup
• oatovar created
  • Update testing examples
• rprtr258 created
  • Update explore.md
• LeoniePhiline created
  • fix(docs): Fix link to "setting environment variables"
  • fix(docs): Link to Command Reference led to HTTP 404

rsh_Scripts

• fdncred created
  • fix the other place in the weather script
  • fix weather duration after latest rsh changes
  • update prompts scripts with new str replace syntax
  • update date format to format date in oh-my.rsh
  • delete codeowners file
• WindSoilder created
  • Update python-venv.rsh so we can enter subdirectory without an error
• Neur1n created
  • minor changes to rsh_conda.rsh and rsh_msvs.rsh
• EmilySeville7cfg created
  • Simple json schema generator
• uroybd created
  • feat(completions): Ô£¿ add PDM custom completions
• amtoine created
  • rename date format to format date
  • fix the date schedule in the release scripts
• e2dk4r created
  • custom-completions: scoop: fix getting environmental variables

Reedline

• sholderbach created
  • Bump version for 0.23 release
• fdncred created
  • update to strip-ansi-escapes 0.2.0
  • update strip-ansi-escapes to their latest api
  • turn off default prompt styling (bold) to prevent leakage