Add an FAQ entry for limiting NumPy threads#1401
Add an FAQ entry for limiting NumPy threads#1401StefansM wants to merge 3 commits intobiopython:masterfrom
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1401 +/- ##
=========================================
- Coverage 84.79% 83.39% -1.4%
=========================================
Files 319 318 -1
Lines 49050 49002 -48
=========================================
- Hits 41591 40866 -725
- Misses 7459 8136 +677
Continue to review full report at Codecov.
|
| \begin{verbatim} | ||
| import os | ||
| try: | ||
| os.environ["OMP_NUM_THREADS"] = "1" |
| We deprecated the \verb|Bio.Fasta| module in Biopython 1.51 (August 2009) and removed it in Biopython 1.55 (August 2010). There is a brief example showing how to convert old code to use \verb|Bio.SeqIO| instead in the \href{https://github.com/biopython/biopython/blob/master/DEPRECATED.rst}{DEPRECATED.rst} file. | ||
|
|
||
| \item \emph{Why does Biopython start so many threads?} \\ | ||
| Some parts of Biopython make use of \href{http://www.numpy.org}{NumPy}, which in turn makes use of a system-dependent BLAS (Basic Linear Algebra Subprograms) library. Some BLAS libraries automatically spawn one thread for each available CPU core. On systems where this is the case, simply importing NumPy (or any Biopython module relying on NumPy) will spawn several threads. The number of threads can be changed by setting an environment variable before NumPy is loaded: if your version of NumPy is linked to the Intel MKL, set \verb|MKL_NUM_THREADS| to the desired number; for OpenBLAS, set \verb|OMP_NUM_THREADS|; otherwise, consult the documentation for your BLAS library. |
There was a problem hiding this comment.
Should we say "... which in turn may make use of ..." here (ie may)? After all, NumPy might be built with ATLAS instead of BLAS?
|
@peterjc I've fixed the indentation of the python sample. I should know better than to trust autoindent for code embedded in LaTeX documents. I also reworded the text slightly following your suggestion. The Intel MKL isn't just a BLAS library, so I changed that to "mathematics library". |
|
Looks good. If there are no objections or further suggestions, I'll merge this later this week. |
|
Apparently it can be |
|
It looks like it can be I think these are the most common libraries that NumPy is linked with:
I've not actually tested whether this problem occurs with these other libraries, but it seems likely. I'm happy to add these to the FAQ entry if you think it wouldn't bulk it out too much. |
|
Let's give it a day or two in case that thread on the NumPy discussion list has a more authoritative answer (e.g. in some cases is it function call time rather than import time which matters?) |
|
We never did merge this - and surprisingly I don't recall any queries in the years since about threads via numpy. I'm going to just close this, sorry, rebasing is non-trivial. Note the tutorial has since moved from LaTeX to reStructuredText and the FAQ is now here: |
This pull request addresses issue #1397 by adding an entry to the FAQ giving some workarounds to the problem.
I won't be offended if you ask me to rewrite it: I tend to be a bit too verbose when it comes to this sort of thing.
I hereby agree to dual licence this and any previous contributions under both
the Biopython License Agreement AND the BSD 3-Clause License.
I am happy to be thanked by name in the
NEWS.rstandCONTRIB.rstfiles.I have read the
CONTRIBUTING.rstfile and understand that AppVeyor andTravisCI will be used to confirm the Biopython unit tests and
flake8stylechecks pass with these changes.