From pipelines to graphs Escape the tyranny of the shell’s linear pipelines with dgsh
160 views · Published 10 March 2018 · 16:02 · Indexed 16 November 2026
Channel: FOSDEM · 2018 · Science & Technology
by Diomidis Spinellis At: FOSDEM 2017 The Unix [dgsh](http://www.spinellis.gr/sw/dgsh/) shell provides an expressiveway to construct sophisticated and efficient data processing pipelines usingstandard Unix tools, as well as third-party and custom-built components. Dgshallows the specification of pipelines of non-uniform non-linear operations.For example _tee_ can feed three processes whose output can then be collectedby _paste_. The pipelines form a directed acyclic process graph, which istypically executed by multiple processor cores, thus increasing the task'sprocessing throughput. We will see how to use _dgsh_ in practice through anumber of general data processing and domain-specific examples, and how toadapt tools for use with _dgsh_. Room: H.2215 (Ferrer) Scheduled start: 2017-02-04 16:00:00