Refactor and overhaul the test suite#3426
Merged
Bibo-Joshi merged 83 commits intomasterfrom Feb 11, 2023
Merged
Conversation
This commit additionally changes: - The scope of the fixtures to module - Class variables are moved into a namespace class called as Space - Auto addition of flaky and no_req marker (in conftest.py) - Caching of default and tz bots (in conftest.py) Apart from this, no tests were added/deleted/modified in any way.
Also adapt the bot tests for this change + removes debug prints + fix few tests
Also change the scope of some fixtures
This mainly does two things: - Leverages the session bot while making new applications, this eliminates all calls to get_me - `test_signal_handler` was optimized to use `loop.call_later` which eliminates the chance where we could get a TaskException not retrieved error, which often fatally crash pytest. These two changes thus reduce flakyness and improves speed.
The last commit addresses the root cause of its addition, so now we don't need this :)
Member
Author
|
These tests don't fail for me locally, will have a look later. I suspect it is due to the change of scope of those fixtures. Edit: See commit message below for details on why it failed. |
This was done as when running pytest with -x, this error is emitted. Not doing this seems to fix the problem. However this could backfire on python versions older than 3.10.6
this is based on whether the test calls get_me indirectly or not
…tinefunction The limitation of that function is fixed by using the `__wrapped__` attribute
…d/create_task let's take full advantage of the lib now being async
includes removing unused params, removing unnecessary monkeypatch.delattr's, missing import, beautification, etc
…llback tests This means we can now move those methods back to TestBotNoReq, since they did not make any request apart from delete_webhook()
The tests are roughly arranged as follows: - test_slot_behaviour - test de_json/to_dict - test init - test equality - test wrong init - etc
Also add a missing test_to_dict in user.py
Bibo-Joshi
requested changes
Jan 22, 2023
Member
Bibo-Joshi
left a comment
There was a problem hiding this comment.
I reviewd the commits since my last review. In a few places it looks like you duplicated an already exsting tests. Maybe those are merge errors or maybe it only looks that way if you don't view all commits …
Member
Author
I think it looks like that since I had to rearrange those tests into these classes. Nevertheless i'll check again |
specially on python 3.7. The xdist_group markers are kept so we can use them locally
Also increase loop.call_later call by 0.1 seconds in test_signal_handlers
Member
|
Awesome work! 💪 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3407. I did not include the
reqmarker, but if you'd like that can be easily added as well.Also some new "features":
Dict(Ext)Botnow overridesget_meto return a cachedUserobject, which is obtained from thebot_infofixture. This saves lots of requests to the realget_meevery time we initialize a bot, app, or updater.Many tests were optimized as well, which should decrease the flakyness in the CI.
How do I review this?!
It is best to review by commit, the changes are specific and I rebased multiple times to make sure changes fit in a certain category.