Along with the release of our brand new API, we also released the source of Flowdock-text, a JavaScript library for linkifying URLs and hashtags. Flowdock-text is built upon Twitter’s awesome twitter-text-js. In this post I’ll explain the rationale behind Flowdock-text, what changes we made to twitter-text-js and present some usage examples.

Parsing URLs from text is certainly no trivial task. We’ve had our problems in that front before. If it wasn’t unmatched parentheses being captured (like this https://www.example.com), it might just have been a case of catastrophic backtracking on certain browsers. In any case, failing to properly turn URLs to links is something that is immediately obvious to the end user, and we wanted to get rid of that once and for all.

Enter twitter-text

Twitter-text knows how to turn URLs, hashtags, mentions and lists into HTML links from any string you give it. Also provided are methods to simply extract them into an array like so:

> twttr.txt.extractUrls("(like in this http://www.example.com)");
[ 'http://www.example.com' ]
> Wow it works great!
SyntaxError: Unexpected identifier

The functionality was exactly what we needed. After all, we sort of do the same thing as Twitter in that regard: users enter plain text, we have to present it in a nice way.

Obviously the twitter-text library is built for the needs of Twitter, and some of its functionality wasn’t a very good fit for Flowdock. Some parts we didn’t need at all (for example list parsing) and some of it just didn’t quite work the way we wanted (parsing usernames), so we made some changes to meet our needs.

Unlike twitter-text-js, Flowdock-text allows dashes as a part of a hashtag. Also due to the use case with Github issues and Bamboo builds Flowdock-text allows all-numeric hashtags.

Flowdock needs to match usernames from messages as well. Twitter-text-js provides methods for extracting and linkifying @-prefixed usernames, but due to the nature of Twitter usernames (which can include only alphanumeric characters and underscores) are inadequate for our needs. This lead to us using the same regexp in both hashtag extraction and username extraction.

We also added an option to extract usernames based on an optional whitelist. This is used internally in Flowdock to exclude non-existent @users from turning in to links. On top of that Flowdock-text provides a few new methods to help message processing.

Usage examples

Using Flowdock-text in your own client is very straightforward. The Flows resource returns a list of users belonging to each flow, which can be passed as-is to Flowdock-text methods. In this simplified example, we’ll extract all tags from a new chat message before it is sent to Flowdock. Provided is also a reference to me-user, to make sure that a special :unread: tag is not applied to that user.

> var users = [
  {nick: "Marty", id: 1, disabled: false},
  {nick: "Emmett", id: 2, disabled: false}
];
> var me = users[0];
> FlowdockText.parseTags("@anyone seen this: http://www.bigwaste.com/bttf/twin_pines_mall.shtml #cool", users, me);
[ 'cool', ':url', ':user:everyone', ':unread:2' ]

Some special tags get returned from parseTags(). Since the provided message contains a URL, the method returns a special :url tag. This allows more fine-grained message filtering in Flowdock search. Also since the message contained one of our special @everyone tags (which are @everyone, @everybody, @all, @anyone and @anybody), :user:everyone is also returned. Appropriate :unread: tags with user ids get returned too.

We also provide some helper methods:

> FlowdockText.mentionsUser("hey @Marty, when this baby reaches 88mph yada yada yada", me);
true
> FlowdockText.mentionsAll("can @anyone help me?");
true

More examples can be found in the Flowdock-text README.

So there’s your short introduction to Flowdock-text. The source resides in Github, feel free to poke around!