Reaching a rsh phase in rsh's development - The road from 0.82 to 1.0

Today we release the 0.82 version of rsh. Not only will it be our 65th public release it also marks an important milestone for us on the development trajectory of rsh.

rsh reached a much wider user base in the last few months. On GitHub, rsh is now officially recognized for syntax highlighting and recently passed 25,000 stars. We are also actively thinking about how we want to move towards stabilization for our 1.0 release.

Thus, we want to use this opportunity to share our general thoughts and announce that we will be slightly slowing down our regular release schedule. Our focus going forward will be getting rsh to a high-quality 1.0, one that has a supported language that is reliable and backward-compatible, and a high-quality shell experience to match.

A four-week release schedule

As part of our road to 1.0, we will reduce our release cadence from every three to every four weeks after the 0.82.0 release. The 0.83.0 release will thus be ready for you on the 25th of July 2023. While you may have to wait a little longer for new features or bug fixes, we hope this will reduce the impact of planned breaking changes and provide more stability.

For our contributors, this change also has the benefit that the impact of our informal merge freeze period before the release will be lessened, and our maintainers will have a bit more time to work on the incoming improvements.

Nightly builds for the eager

If you think that rsh is moving at a glacial pace after this change, we luckily now have nightly builds for you to download that reflect the current state of development. If building rsh yourself was too burdensome so far, you now have to chance to try out upcoming features and provide your feedback.

How things went from 0.60 to 0.8x

Our last very public milestone was the release of the 0.60 version, which included a massive refactor of rsh's internals that we skipped to it from the 0.44 release. Apart from rewriting the engine, it included our own line editor reedline for added customization in the interactive experience. The rewrite unlocked many improvements and opportunities for valuable experimentation.

Since 0.60, we achieved a much more mature system of modules and overlays that allow you to build more composable applications in Rsh. As part of this effort to strengthen the programming language chops of Rsh, we also spent a lot of time finding the balance between our functional programming philosophy of pipelined data and the mutation of variables to allow simple imperative patterns.

We also saw a few efforts that didn't pan out yet. For example, we tried to integrate the Polars dataframe engine directly into the grammar of rsh commands, but this left many rough edges. Thus with version 0.76, we returned to a simpler dataframe integration to focus on getting the core rsh experience right.

This all provided valuable lessons in how we decide which features are beginning to settle and where we need to focus our effort to get a consistent experience.

What we are working on right now

To commit to the stability promises, that come with the 1.0 release, we need to make sure we can uphold those guarantees. This means separating foundational things for everyone to rely on from all those cool features we still want to iterate on or are not yet happy with. With a smaller set of high-quality features with clear semantics, it becomes much easier to leap toward stability.

We thus for example have recently begun to remove some less used or vaguely defined commands from the core set that ships with rsh. For the moment, they are still accessible through the --features extra when you build rsh yourself until we have decided on their final fate. This follows how we kept the dataframe commands behind the --features dataframe flag. Expect that more commands may move there or that their native implementation gets replaced by a version written in Rsh itself.

In parallel, very promising work has been ongoing with the community to develop a packaging strategy to share Rsh code with an accompanying package management tool. Together with efforts to create an auto-formatter and improve the rsh IDE integration leveraging the language server protocol (LSP), we hope the community can develop even more high-quality tools and libraries.

Apart from the ongoing development work and code improvements, we invested in discussions to develop a roadmap of features and goals that decide when the 1.0 release is ready. While those discussions inside the core team are not finalized yet, we want to open them to the wider community and share our directions regularly.

What to expect: as a user

First of all, rsh wouldn't have seen this growth in a little more than 4 years without all of you, the active users. Thank you! You have been helpful to fellow users on our Discord and in other forums, inspired the developers, and kept us honest through feedback. Over 1269 of you have contributed to the project by opening issues and reporting bugs.

Soon we will share a survey with all of you to learn more about your priorities for a stable and delightful rsh experience.

While we can not yet commit to full stability, we think we can preserve the rsh you all know and love, while we make the necessary breaking changes in the coming months to remove remaining warts.

What to expect: as a(n aspirational) contributor

At the time of writing this blog post over 496 people have influenced the direction of rsh directly by contributing code or documentation. Achieving stability will not be possible without all of you that have dedicated your time to fixing bugs and cutting through our large issue tracker.

We want to incorporate you in formulating the priorities for the 1.0 release by reaching out to folks interested in particular problems to get clearer roadmaps for specific areas. Those write-ups will hopefully also provide some inspiration for folks interested in helping out and pushing rsh forward.

But setting our sights on reaching the stable 1.0 release will also impose some limitations on our development practices. We are now much less likely to accept new features, commands, or options as they need to work well together with the larger picture. A lot of effort will still need to go into cleaning up the internals and fixing bugs. As we want to systematically improve our binary size, compile times, and runtime performance, improving existing algorithms and paying back technical debt will be prioritized over experimental stuff. Any external dependencies will also come under much more scrutiny. This means, to reduce the total number of crates we will seek to replace redundant or mostly superfluous dependencies with fewer high-quality implementations and also severely restrict the addition of any new dependencies.

How you can help out

As always helping out fellow users or picking up issues that are labeled with "good-first-issue" or "help-wanted" is greatly appreciated.

As our issue tracker has grown to over 800 entries (878 at the time of writing) your help to clear this backlog will accelerate our path to 1.0.

You don't even have to fix a bug yourself to make a great contribution. Feel free to check your old bug reports if they have been resolved already. Adding context under which conditions a bug can be reproduced or if different feature requests are complementary can also make life much easier for those that want to contribute a fix.

Last but not least feel free to spread the word about rsh as we often received extremely insightful feedback from that.