Over the past several months, we’ve been hard at work on an application that leverages Twitter to both aggregate and deliver targeted data to thousands of users in real time. Throughout the day, data is scraped, parsed, and turned into actionable information. While stand alone application development has its own rewards and challenges, integration with a 3rd party API adds another layer of complexity. Below is a short summary of some issues to keep in mind as you develop your Twitter app. These tips focus primarily on applications that require two way communication, but can be applicable in other scenarios as well.
1. API Updates and Oauth
While Twitter’s API is relatively simple to work with, Twitter is notoriously vague about Oauth and API changes. More importantly, it will not always be clear how these changes will effect your application. You should be mindful of these changes and build flex time into your timelines to account for these variables. When creating the app specifications, it may be helpful to ask your client to subscribe to the relevant Twitter channels and become generally familiar with the API. This will undoubtedly save both parties time when problems arise, and get everyone on the same page.
Use @Twitter and the wiki to stay updated.
2. White Listing and API Calls
Like many services, Twitter limits the amount of API calls an IP or account can make per day. Twitter does offer white-listing, but this will be of little help during the early stages of development. If you are scraping for specific data, it may be wise to initiate automated processes by hand to avoid exceeding your rate limits. Once you have decided that your process is solid and you are ready to implement routine scraping, start with a small funnel and test. Then open up the funnel when your rate limits increase. Exceeding your API call limits can cause your IP to be blacklisted and force you to find temporary alternative solutions. A Twitter Sandbox may be in the works for V2 of the API.
3. Integrity of Data
Twitter’s data is loosely organized and follows no true uniformity. This increases the importance of understanding the correlation between how you collect, validate, organize, and present information. As you design and develop, your creative team and technical lead must make sure they understand how the data is collected and what information can be reliably presented to users. Automated scraping and delivery processes must be streamlined to focus on only the most relevant, verifiable, and timely data. If your application simply takes a fire-hose approach (accepting Tweets even remotely relevant) the value of your data will drop significantly and your users will move to a better application that offers the same benefits. While this approach may mean that some valuable data is ignored, it is important to realize your application’s true value is organizing and delivering Twitter’s enormous data stream succinctly and without garbage. Refining this data is a continual process, as Twitter’s use and conventions will undoubtedly continue to evolve and change.
4. Real Time Delivery
Clearly the true value of Twitter is its ability to deliver large amounts of data in real time. When developing, it is important to keep this in mind. Relevant Twitter content must be delivered as instantaneously as possible. If your cron jobs take an exorbitant amount of time to gather, process, and present content your applications value will dip. Project managers and clients should agree on acceptable time periods for when data will be processed and delivered.
5. Leverage the Community to Train New Users
Twitter users have a unique ability to train themselves to use the new syntax and shortcuts. Use mentions, DMs, and updates to provide clues on the best way to use your application. For example a “Welcome” mention may include a sample command. Twitter users utilize several shortcuts, when possible these shortcuts (or something similar) should be integrated within your application. This will increase the likelihood of proper use and increase adoption rates. You can find the official Twitter lingo here.
