Comments on: Bugs in Twitter Text Libraries

By: Matt Sanford

Matt Sanford — Wed, 31 Mar 2010 15:36:20 +0000

It’s great to have people checking over the auto-linking issues at last. The http://www…foo issue was reported at http://github.com/mzsanford/twitter-text-rb/issues/closed#issue/4 and has since been fixed in the Open Source library and is awaiting deploy on twitter.com.

It turns out to be a harder problem that expected. The http://www…foo bug was introduced while adding support for some IDNs. I’ll investigate the TLD stuff but the main worry is the gTLD process and what that will mean for the list of valid TLDs.

By: Steven Pears

Steven Pears — Wed, 31 Mar 2010 13:01:09 +0000

I think the regex is a good start, but that you need to make it the first in a two-pass process. Put a word boundary on the regex so that it doesn’t allow for “awwww…”, then use regex groups to give you a quick and easy way to check a valid TLD.

If your TLD list is out of date then you accept you may not catch 100%, but you’re going to get a much more accurate set of results compared to regex alone, and in the long run people will appreciate it.

By: David Carrington

David Carrington — Wed, 31 Mar 2010 11:55:35 +0000

My point is that the list of TLDs changes, and I can’t be arsed to subscribe to something which notifies me when I need to refresh the list in my apps. It will also make the regex even bigger than it is now, which I’d personally prefer not to do.

There is also the problem with clashes such as “Hi john.in tomorrow?” – is that http://www.john.in? I think not. It’s those kinds of issues which I believe are not worth the effort of solving for the sake of 7 characters.

By: Tweets that mention Bugs in Twitter Text Libraries – Terence Eden has a Blog -- Topsy.com

Wed, 31 Mar 2010 11:53:28 +0000

[...] This post was mentioned on Twitter by James Heaver. James Heaver said: RT @edent: Bugs in Twitter's text processing libraries – http://shkspr.mobi/blog?p=1924 (thoughts from devs welcome) [...]

By: Owen Blacker

Owen Blacker — Wed, 31 Mar 2010 11:32:05 +0000

I’d disagree with David about adding support for the full canonical list of TLDs — it’s software, it’s not as if it’s hard and a 5% character saving is gonna be useful to quite a few people.

But you’re both definitely right about consistency, that’s just sloppy of them!

By: David Carrington

David Carrington — Wed, 31 Mar 2010 10:48:10 +0000

Consistency I agree needs to be fixed. The www problem too.

I’m not convinced that adding handling for all TLDs is worth the few extra characters saved.