Skip to content

from zero to hero with rust: build a blazing-fast cli tool and crush i/o bottlenecks in minutes

why rust for cli tools?

rust empowers beginners and seasoned engineers to build fast, safe, and reliable command-line tools. its zero-cost abstractions and fearless concurrency help you crush i/o bottlenecks without sacrificing readability. if you’re coming from full stack or devops, rust’s tooling, package manager (cargo), and strong community will feel refreshingly productive.

  • performance: native speed with zero-cost abstractions.
  • safety: compile-time guarantees prevent data races and many runtime crashes.
  • portability: generate a single static binary that’s easy to ship in ci/cd pipelines.
  • great tooling: cargo for build/test/publish, rustfmt for formatting, clippy for linting.

what we’ll build

a blazing-fast cli that scans files and directories, counts lines, words, and bytes (like a mini wc), and supports:

  • recursive directory traversal
  • parallel processing for i/o-bound workloads
  • globbing (file patterns)
  • colorful, human-friendly output

along the way, you’ll pick up skills useful for devops, full stack, and seo-minded content workflows (e.g., batch-analyzing content repositories, logs, or sitemap exports).

prerequisites

  • install rust and cargo: curl --proto '=https' --tlsv1.2 -ssf https://sh.rustup.rs | sh
  • familiarity with basic terminal usage
  • optional: ripgrep mindset—search large codebases and logs with speed

project setup

# create a new binary project
cargo new fastwc --bin
cd fastwc

# add helpful dependencies
# - clap: cli args
# - walkdir: directory traversal
# - rayon: parallelism
# - globset: fast glob matching
# - colored: colored output
# - anyhow: ergonomic error handling
cargo add clap --features derive
cargo add walkdir rayon globset colored anyhow

designing the cli

we’ll design a simple interface:

fastwc [options] <path>...

options:
  -r, --recursive     recurse into subdirectories
  -g, --glob <pat>    only include files matching glob (e.g., **/*.rs)
  -p, --parallel      enable parallel processing
  -q, --quiet         less verbose output
  -h, --help          show help

src/main.rs (minimal version)

use std::{fs::file, io::{self, bufread, bufreader}, path::path};
use clap::{arg, command};
use walkdir::walkdir;
use globset::{glob, globsetbuilder};
use rayon::prelude::*;
use colored::*;
use anyhow::{result, context};

#[derive(debug)]
struct counts {
    lines: u64,
    words: u64,
    bytes: u64,
}

fn count_file(path: &path) -> result<counts> {
    let file = file::open(path).with_context(|| format!("open {:?}", path))?;
    let mut reader = bufreader::new(file);
    let mut buf = string::new();
    let mut lines = 0u64;
    let mut words = 0u64;
    let mut bytes = 0u64;

    loop {
        buf.clear();
        let read = reader.read_line(&mut buf)?;
        if read == 0 { break; }
        lines += 1;
        words += buf.split_whitespace().count() as u64;
        bytes += read as u64;
    }

    ok(counts { lines, words, bytes })
}

fn build_globset(patterns: &vec<string>) -> result<globset::globset> {
    let mut builder = globsetbuilder::new();
    for p in patterns {
        builder.add(glob::new(p)?);
    }
    ok(builder.build()?)
}

fn main() -> result<()> {
    let matches = command::new("fastwc")
        .about("blazing-fast line/word/byte counter")
        .arg(arg::new("path").required(true).num_args(1..).help("files or directories"))
        .arg(arg::new("recursive").short('r').long("recursive"))
        .arg(arg::new("glob").short('g').long("glob").num_args(1..))
        .arg(arg::new("parallel").short('p').long("parallel"))
        .arg(arg::new("quiet").short('q').long("quiet"))
        .get_matches();

    let recursive = matches.get_flag("recursive");
    let parallel  = matches.get_flag("parallel");
    let quiet     = matches.get_flag("quiet");
    let globs: vec<string> = matches.get_many::("glob")
        .map(|vals| vals.cloned().collect()).unwrap_or_default();

    let globset = if globs.is_empty() { none } else { some(build_globset(&globs)?) };

    let paths: vec<string> = matches.get_many::("path")
        .unwrap().map(|s| s.to_string()).collect();

    // collect candidate files
    let mut files = vec::new();
    for p in paths {
        let meta = std::fs::metadata(&p).with_context(|| format!("metadata {}", p))?;
        if meta.is_file() {
            files.push(std::path::pathbuf::from(&p));
        } else if meta.is_dir() {
            if recursive {
                for entry in walkdir::new(&p).into_iter().filter_map(result::ok) {
                    if entry.file_type().is_file() {
                        files.push(entry.path().to_path_buf());
                    }
                }
            } else {
                for entry in std::fs::read_dir(&p)? {
                    let entry = entry?;
                    if entry.file_type()?.is_file() {
                        files.push(entry.path());
                    }
                }
            }
        }
    }

    // apply glob filtering
    if let some(gs) = &globset {
        files.retain(|f| {
            if let some(s) = f.to_str() {
                gs.is_match(s)
            } else {
                false
            }
        });
    }

    if files.is_empty() {
        eprintln!("{}", "no files matched.".yellow());
        return ok(());
    }

    let process = |f: &std::path::pathbuf| -> option<(std::path::pathbuf, counts)> {
        match count_file(f) {
            ok(c) => some((f.clone(), c)),
            err(e) => {
                eprintln!("{} {:?}", "skip (error):".red(), e);
                none
            }
        }
    };

    let results: vec<(std::path::pathbuf, counts)> = if parallel {
        files.par_iter().filter_map(process).collect()
    } else {
        files.iter().filter_map(process).collect()
    };

    let mut total = counts { lines: 0, words: 0, bytes: 0 };
    for (path, c) in &results {
        total.lines += c.lines;
        total.words += c.words;
        total.bytes += c.bytes;
        if !quiet {
            println!("{}\t{}\t{}\t{}",
                c.lines.to_string().cyan(),
                c.words.to_string().green(),
                c.bytes.to_string().blue(),
                path.display()
            );
        }
    }

    println!("{}\t{}\t{}\t{}",
        total.lines.to_string().bold(),
        total.words.to_string().bold(),
        total.bytes.to_string().bold(),
        "total".bold()
    );

    ok(())
}

understanding the performance

  • i/o-bound speedups: enabling --parallel uses rayon to process files concurrently, ideal when reading many small files.
  • reduced allocations: reuse a buffer (buf) while reading line by line.
  • zero-cost abstractions: iterators compile down efficiently; no garbage collector pauses.

try it out

# count rust files recursively, parallel
cargo run -- -r -p -g "**/*.rs" .

# count everything in current directory (non-recursive)
cargo run -- .

# quiet mode for scripts
cargo run -- -r -p -q .

devops integrations

this binary is ideal for ci/cd pipelines where you need fast content or log analysis.

  • pre-merge checks: ensure repo size or line counts stay within limits.
  • build metrics: track lines changed per build.
  • container-friendly: ship a single static binary in lightweight images.
# example github actions step
- name: fast line/word/byte metrics
  run: |
    cargo build --release
    ./target/release/fastwc -r -p -g "**/*.{rs,md,html}" . > fastwc.txt
    echo "metrics saved to fastwc.txt"

full stack and seo use cases

  • full stack: measure template sizes, api response samples, or content payloads in development.
  • seo: analyze large content repositories, count words per article, or validate sitemap exports quickly.
  • content ops: batch-inspect markdown/html files for word counts before publishing.
# seo example: count words in content folder
fastwc -r -p -g "content/**/*.md" content/

benchmarking tips

  • use hyperfine to compare against other tools.
  • warm up the disk cache by running once before measuring.
  • benchmark different buffer strategies (e.g., read_to_end vs line-by-line) for your dataset.
cargo install hyperfine
hyperfine './target/release/fastwc -r -p -g "**/*.rs" .' 'wc -l $(git ls-files "*.rs")'

quality and safety

  • clippy: cargo clippy -- -d warnings to keep code idiomatic.
  • formatting: cargo fmt for consistent style.
  • testing: add unit tests for counting logic and integration tests for cli behavior.

example unit test

// add to src/lib.rs if you extract logic there
#[cfg(test)]
mod tests {
    use super::*;
    use std::io::write;
    use tempfile::namedtempfile;

    #[test]
    fn counts_basic() {
        let mut f = namedtempfile::new().unwrap();
        writeln!(f, "hello world").unwrap();
        writeln!(f, "rust rocks").unwrap();

        let c = super::count_file(f.path()).unwrap();
        assert_eq!(c.lines, 2);
        assert_eq!(c.words, 4);
        assert!(c.bytes >= 22);
    }
}

packaging and distribution

  • release: cargo build --release for optimized binaries.
  • cross-compile: use cross to build for linux, macos, windows in ci.
  • versioning: tag releases and attach binaries for easy downloads.
cargo install cross
cross build --release --target x86_64-unknown-linux-musl

common pitfalls and fixes

  • unicode counts: the example uses byte length and split_whitespace. if you need grapheme clusters, use unicode-segmentation.
  • large files: line-by-line iteration is memory-friendly; avoid reading entire files unless necessary.
  • symlinks: decide whether to follow them; walkdir has options to control this.
  • error handling: we skip unreadable files but log errors; adjust for stricter behavior if needed.

next steps

  • add json/csv output for dashboards.
  • expose a library crate for reuse in other rust services.
  • add progress bars with indicatif.
  • implement async i/o for network-bound tasks using tokio.

key takeaways

  • rust lets you go from zero to a production-grade cli quickly.
  • parallel file processing and careful i/o dramatically reduce bottlenecks.
  • great fit for devops, full stack, and seo workloads that need speed and reliability.

cheat sheet

  • build: cargo build --release
  • run recursive + parallel: fastwc -r -p .
  • filter by glob: fastwc -g "**/*.md" content/
  • ci step: compile, run, save report

The views expressed in this post are those of its author and do not represent pilput. See our Terms of Service.

Comments

Share your thoughts and join the conversation.

Loading comments...

Join the Discussion

Please log in to share your thoughts and engage with the community.