Skip to content

feat: add --jobs flag to limit concurrent file imports #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+9 −2
Open

feat: add --jobs flag to limit concurrent file imports #117

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,12 @@ pub struct CommonArgs {

#[clap(long)]
pub show_secret: bool,

/// Number of parallel jobs to use while importing files.
///
/// Defaults to the number of logical CPU cores.
#[clap(short = 'j', long)]
pub jobs: Option<usize>,
}

/// Available command line options for configuring relays.
Expand Down Expand Up @@ -367,8 +373,9 @@ async fn import(
path: PathBuf,
db: &Store,
mp: &mut MultiProgress,
jobs: Option<usize>,
) -> anyhow::Result<(TempTag, u64, Collection)> {
let parallelism = num_cpus::get();
let parallelism = jobs.unwrap_or_else(num_cpus::get);
let path = path.canonicalize()?;
anyhow::ensure!(path.exists(), "path {} does not exist", path.display());
let root = path.parent().context("context get parent")?;
Expand Down Expand Up @@ -711,7 +718,7 @@ async fn send(args: SendArgs) -> anyhow::Result<()> {
)),
);

let import_result = import(path2, blobs.store(), &mut mp).await?;
let import_result = import(path2, blobs.store(), &mut mp, args.common.jobs).await?;
let dt = t0.elapsed();

let router = iroh::protocol::Router::builder(endpoint)
Expand Down