OpenCalais

OpenCalais is a tool created by Thomson Reuters for extracting metadata from documents. It utilizes an engine called ClearForest (a recent aquisiton by Reuters) to run a series of NLP (Natural Language Processing) based text analytics on any content sent to it.

Why is OpenCalais interesting as compared to other tools of its kind?
1. It's free.
2. It can be called as a Web Service.
3. It's open to the public and out there on the interwebs.

Here's an example of how you'd use OpenCalais in semi-real life:

Pretend you're an insurance broker with 1,000 clients (you're doing quite
well). Each transaction you make with a client produces a document of some
sort, which you've been storing on your intranet. Someone asks you this
question: How many of your clients have insured properties in Paris, France over
the years? How would you get an accurate count? This information is certainly in
the documents you've created, but it would take hours of relentless searching to
get an answer. So we end up either not knowing statistics like this - or paying
for expensive software to make us enter this information during our workflow.

This is a trivial problem for OpenCalais. OpenCalais extracts entities like people, places, and companies right out of the box. You send it a document, it sends back all those entities. Simple! Our insurance broker could easily sort his documents by city, person, or company. It does more than just people and places, though, there are over 100 types of entities that OpenCalais looks for by default. (From movies, to stock splits, to medical conditions). Thomson Reuters has chosen entities that will be useful to professionals from various industries.

Now we get to the part about what makes OpenCalais special. There are other engines out there that extract metadata from documents, sure. But OpenCalais can do this on the fly as a web service. Imagine if every time you wrote a word document it extracted the relevant entities for you and presented you with a list. You could tell it to automatically create links to any company names. Perhaps you're a doctor and every time you write a client's name in a document you'd like it to automatically link to their medical records.

I like the idea of on the fly metadata extraction. Even now, as I write this blog, I'm wondering how I will tag it. Hmm.. if only I had some sort of way to automatically extract entities from my documents and create tags.

What would you use it for?

Agile in the Sales Field

Read on to find out how to extend Agile all the way to the customer.

A seminar on Lean Manufacturing, and sessions on Agile Software Development have convinced me it would be beneficial to "Agilify" our team of Applications Integrators here at Thomson Reuters.

So I joined an internal Agile group, got in touch with Agile coach Paul Ellarby, and started to learn as much as I could.

One of the first events I attended was about gathering and writing user stories. I was inspired by the approach; keeping close to the customer's actual needs and wants was a strategy that resonated with me deeply. While Paul spoke about the techniques he prefers to utilize when ellicting and creating user stories, I began to think:

"Why couldn't user story gathering happen in the field, where the customer is actually located?"

As a large corporation we have many direct points of contact with our customers, none of which are gathering requirements continuously. I've come to believe this is an oversight that many companies are making who are adopting Agile and other "customer-centric" or "customer front-end" strategies, but failing to really meet with the customers on a continous, iterative, cyclical basis. These practices are paramount to Agile, yet few in large corporate environments seem to be taking them all the way to the customer.

Do we have any employees in the field that continuously meet with the customer, (ideally) understand their needs, and in general are challenged think from a customer's perspective? The sales professionals. That's their exact job. So, without further ado, here's the theory:

Steps for Extending Agile to the customer (Agile Product Development):

1. Utilize Sales / Account Management teams to close the feedback loop, gathering stories directly from the end user.
2. User stories are sent into an aggregation process. Filtered, interpreted, by Product Development teams.
3. Product Development Teams organize stories in the normal way, communicating with the Software Developers in a traditional Agile method.
4. End of release cycle, product is taken to the customer to check for satisfied needs. In this case the customer is the real customer, not necessarily a product development group.

This is a rough draft of the theory. Currently I'm endeavouring to organize a pilot group willing to test this out. I'll be documenting the results as we proceed.

How close is your development team to the customer? I'd be interested in hearing your perspective as a Software Engineer, Product Developer, or anyone involved in a software development process.

Applications Integration

In January 2008 I took on a newly formed role at Thomson Reuters titled Applications Integrator. On paper, this role appeared to be a hybrid customer facing and programming role. This was a good fit for me because those were the capacities I'd served over the last few years.

Upon starting the role it became clear that there was going to be no top-down direction. My team of, well, to begin with, just me (later we expanded to a whopping four individuals) was basically let free to scamper off into the night with little instruction. This wasn't anyone's fault, they were having trouble filling the leadership position above us in the chain.

So, I began to try and build an image for our team. I could try to lead them directly, but there was so little pressure on the team it made motivation near-impossible. So I thought hard about why the company created the role and especially the customer need they were trying to satisfy.

Custom software, that was a huge part of what they expected. I believe they wanted to increase both customer satisfaction and the extent of which our software is embedded into the customer work flow. When friends asked me what I did for a living, I found myself describing it as:

"Meeting with our customers, large law firms, to determine where the software we create is not fulfilling their needs. Then, where appropriate, developing custom software to fill in the gap."

I've since increased the accuracy of this vision but I believe the sentiment remains the same.

This blog is about how this vision evolves over time and the methods we've adopted and created to better serve our customers from a technology integration standpoint. Further entries will talk more specifically about the tools, technology, and methods we find useful.