What about to include also captions in the helper using regex?

@91DarioDev Good point.

@91DarioDev

• Extracting URLs from Captions

Fails on all except py3.6 due to urllib.parse

@JosXa Can you please make sure the PR passes pre-commit and tests?

test_helpers succeeds in all test cases. However, unrelated tests fails. So lgtm, but @Reviewer please have another look at the tests.

@JosXa Can you remind me why we would need/want this? And anyway I think it's more logical to sort in order of appearance rather then alphabetical? Also to parse them from captions please use the parse_caption_entites

Turns out the issue was hidden in the call of message.parse_entities. This builds a dict which is not sorted in python <=3.5. So now after parsing, the entities are just sorted.
One thing I want to mention: The method now can only handle single trailing slashes, i.e. [github.com/, google.com, google.com/ github.com//] will produce [github.com/, google.com/ github.com//]. However I guess trailing double slashes are very rare (if even existent?), so that should be ok IMO …

PS: AppVeyor fail is unrelated ;)

@Bibo-Joshi I'm not quite sure why you would want to filter this at all. The difference between having a trailing slash or not in a URL is not necessarily insignificant.

Unrelated, the comment in line 185 says "dublicates" instead of "duplicates".

@jh0ker I guess that 9-9-19 me interpreted @nmlorg s comment such that in doubt it's better to have a trailing slash than not. But neither 9-9-19 me nor now-me have much expertise on that area (or any area at all 😅) …

@SpEcHiDe brought up an interesting point. He says that he's using something like this in his bot:

Maybe we could also provide some abstractions over join links, message references, and t.me URLs 🤔

Test fail is unrelated

After long considerations and real intent to see this PR merged, we, the maintainers, decided that this PR is too specific to be included in a general purpose library.
Therefor we've decided to close the PR without merging.

For future reference: this will be made available through the new ptbcontrib library