tr: Stream output instead of buffering#6012
Merged
sylvestre merged 1 commit intouutils:mainfrom Mar 10, 2024
Merged
Conversation
|
GNU testsuite comparison: |
3dc2b0f to
9af8eda
Compare
Collaborator
Author
|
Changes since last push:
Test output of flaky test `test_split::test_round_robin_limited_file_descriptors`Sadly, this does not show a backtrace of the failing command. Also, I cannot reproduce this problem. On my machine the command only needs 5 open files at any point in time, so |
|
GNU testsuite comparison: |
sylvestre
reviewed
Feb 28, 2024
9af8eda to
86b12a5
Compare
Collaborator
Author
|
Changes since last push:
|
86b12a5 to
55eee60
Compare
Collaborator
Author
|
Changes since last push:
|
Closed
Collaborator
tertsdiepraam
left a comment
There was a problem hiding this comment.
I like it! Just one suggestion on the link in the comment.
9127b4b to
8898ba8
Compare
Collaborator
Author
|
Changes since last push(es):
|
This should lower memory consumption, and fixes OOM in some scenarios.
|
GNU testsuite comparison: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should lower memory consumption, and fixes OOM in some scenarios.
Before, buggy behavior:
After:
It might be tempting to factor out the common line
translate_input(&mut locked_stdin, &mut buffered_stdout, op);from all if-branches. However, note thatophas distinct types in each case, and virtualizing it (e.g.let op: Box<dyn SymbolTranslator>) would incur additional runtime costs. That cost seems unnecessary just to gain a little bit of beauty.I'm not really sure how to test this. Ideally we could pipe in some data, and NOT close the stdin pipe, and see whether the program produces any partial output. However, I didn't see any methods/functions for doing so, and it sounds like it's too hard to get right to be worth it.
Maybe I could write a
BENCHMARKING.mdfile, like forshuf? What do you suggest?