This article was updated on June 4, 2017

Developing Your Short List

Evaluating time and expense tracking software can be a daunting task given the number of options out there. You begin thinking of the steps: "I'll read some white papers, look at screen shots, develop a short list, schedule demo meetings and do some free trials." Your thinking is right on track. Then you're off to Google or Bing find the information you need so you can narrow things down. But when you do search one the phrase "timesheet software," Google says there are 10,000,000 pages to look at. You start to twitch as your cortisol levels start to soar. Then one of two impulses will take over: 

Flight:
"Maybe we can get more life out of our paper and Excel sheets !" 
"Maybe that Lotus Notes app we built 11 years ago will be okay after all!"
"Screw this, I'm going to lunch!"

Fight:
"10,000.000 pages? Really?"
"I've got to narrow this down"

As you fight through this commend yourself as you push ahead because you're on the right track. Evaluating software, products and services that are delivered in a highly fragmented industry, are identified and sourced, though not necessarily purchased, almost entirely online. They are after all made of bits and bytes so why would you find them in a print ad. These days that would be like going to the moon to buy a good avocado.  

A Software Industry Primer

Industry Structure.
Sit down. Get a cup of coffee for a short history lesson on the software biz.
During five crazy-assed years in the early aughts (2000's), Wall Street and Silicon Valley lost their way, and years later you, as the poor software evaluator, have to navigate through the aftermath. The story starts with the US economy which had been on the skids after the 1980's asset selloff. In 1994, Netscape, the first browser company, went public and used a very crude browser aptly named “Netscape” to give a new life to the old Internet. "Domain names," Internic-speak for "URLs," were bought up in a modern-day land grab. Millions, that's millions with an M, were created and slapped on web sites, or parked somewhere for the day when an investor might get drunk one night and want to buy:

"www.designericecubetrays.com" or 

"www.webbabystuff.com"

For every Google or Amazon there were a startling number of companies like WebVan and eToys. While online fashion site Boo.com was handed $187 million to blow through in one year, Wall Streeters also put crazy valuations on so-called "web development" companies. These were actually hyped-up consulting firms building what are now redumentary web sites for eye-popping fees. Stuff you would pay a teenager to do today. Later on, these web development guys eventually developed the early website software widgets, navigation tools, e-commerce platforms and other stuff too boring to write about. There were many people at every level of the Internet business.

Around 2003, the tech bust finally came crashing down, the web development guys suddenly had lots of free time. And, you guessed it, they started launching a lot of web pages and niche software products. In some cases, many of these companies had already developed their own internal applications prior to the crash: timesheet software, customer relationship management systems and other web-based applications. So, as the tech downturn carnage began, a normal brain would think the number of software applications would start dwindling down. But no! The exact opposite happened! They exploded! Everybody who had an application setup a web site, with or without customer lists, and began selling.
The result: seemingly hundreds and thousands of companies popped up out of no where like tall weeds on the software landscape. This had made it harder for you to pick your one beautiful flower; the software application that is the close or exact fit for you and your company. 

Out of all of this craziness, there are many many companies, great at developing websites, maybe even great at SEO and advertising, that look very real. But, now you are warned, many are without many customers, without much industry knowledge, and they stand there tall and proud on the Google search results pages ready to confuse the hell out of you. 

Evaluation Criteria

Here's the good news. Now that you know this story, you can be extra smart and add these items to your evaluation checklist:

Customer references

Always insist with any vendor, before you make a final purchase, that you speak with two or three relevant references, that can talk about their experiences implementing and using the application.

Customer lists

A solid successful company that has a strong customer history will have many hundreds of customers. You want to see, or be able to talk to customers who are like you. If you are a large company with a large number of users, you want to see customers who are look like you. If you have particular requirements that are peculiar to your industry, or related industries, you want that. 

Integration capabilities

Having the application be able to talk with your payroll, accounting or HR systems will have significant impacts on your return on your application investment.

Flexibility

If the application is not flexible, when your company grows, acquires new units or changes significantly you will being kicking the rigid application to the curb.

System standards

A demonstrated history of supporting all major browsers, apps, operating systems, security standards and more means that this application (and the company behind it) will be able to evolve and grow with the times.

Multiple User Interfaces

As things change, user interfaces should become better and more efficient. Your chosen application needs to demonstrate a strong track record of providing as many user interfaces have needed in the past, as an indication what they will do in the future.

Great Support

An application is only as good as the support you receive when you have questions or issues arise. Some of the most serious issues with software applications are not bugs, but rather how changing setups and configurations can cause problems if not performed optimally for your company.

Pricing

You want the application priced to be competitive with other leaders in the space, but you don’t want it to be 30, 40 or 50% less, because that company is either cutting corners or not going to be around in five years.

Software Trials

A free trial should be mandatory because it is not really free, there is a major cost to the time and effort of you and your colleagues to evaluate the trial software. The trial software should be fully featured, not a “light” version. You should also be able to test every feature and utility in the system that you might need. It's easy to purchase a system based upon beautiful-looking Powerpoint presentations, sales people waving their arms around over-committing and overselling. Insist on a software evaluation where you get to kick the tires and test your most important scenarios.

Statements of Work

Your company is special and has particular ways of doing things, and there can be gaps in the feature sets of even the best software applications. You also might want to move forward with a vendor because they have several other compelling features. In these cases, you should always have the vendor of choice provide a detailed SOW (Statement of Work), including budgets and timelines for the development and completion of any features or enhancements needed to address requirements gaps. Insist on this. Make sure to review this SOW thoroughly. Make it addresses what you need. For the important features, it should include detailed wire diagrams, detailed functional descriptions of features and details on how it will serve your most important use cases and user scenarios. The SOW should also be part of your larger business services or licensing agreement.   

Budget Quotes

You might need to see pricing for different scenarios, including different volume purchasing levels, with or without certain integrations or new features. You might need these for next year’s budget well in advance of your conducting demos or a full evaluation of an application. Whatever the reason, the vendor should provide budget quotes in a professional and prompt manner, even if the project has not been approved in your company.      

Implementation Capability

This is difficult to assess except through asking thorough questions of reference customers. You might also gain insight during a trial evaluation into the vendor's ability to setup and modify the settings of their applications. Again, another reason to do a full trial evaluation. Current-day software trials, with more advanced configuration interfaces and tools, allow you to use the trial experience as a proxy for how the full implementation is likely to go.

Stability

In software vendor stability is difficult to assess. It’s good that a vendor has been around the block a few times. On the other hand, it’s not good if it’s long in the tooth and promoting technology that’s so 1999. If you are in an older company that's been around for sixty years, looking for web-based software vendors that are half your age will yield a short list of no one. A software company can be a only a few years old and not be a risky selection if they are well-funded and selling a much better mousetrap. On the other hand, vendors, even with serious-looking backers, selling things on the cheap to gain a lot market share are likely to end up as 1) a vendor who one day seriously raises their cheap prices to monitize the investment of their increasingly cranky backers or 2) a vendor who goes down in flames as its backers stop funding its strategy of predatory pricing.  

Given these evaluation criteria, you can now start more confidently developing your “short list.” 

A few tips on finalizing your short list

You don't want your goal of developing the short list to be so focus on finalizing the short list, that you do not pick the best vendors for your company. Rush things and the quality and appropriateness of the vendors on your short list might be questionable. If every vendor looks the same, and there are so many of them, it is difficult to narrow your list down to the most competitive and high-quality vendors. Here are some tips to keep in mind:

• Try to withstand the pressure to shorten your short list too quickly, though you might be under a deadline.
• Try to prioritize the most important selection criteria and use them so you stay lazer-focused on what's most important as you shorten the list.
• If you selection criteria are not clear, or you do not have enough consensus on this, maybe get approval to start over until you have clear criteria that your most important stakeholders can agree on. Short changing things here can lead to spending a lot of time on meetings, evaluations and gnashing of teeth for a selection process that is aborted later.
• Do call references, and do give them ample time to get back to you. If they are legitimate references they will be busy like you are.
• Don't allow a salesperson to make you add one of his only differentiating features as a "must have" if you don't need it at all. Feature lists are helpful when presenting things that you need now or in the future. But don't allow a valuable-sounding widget to be weaponized and kill off other great vendors deserving of your serious consideration.
• Make sure budget quotes come back to you in a reasonable amount of time.
• However, if you are asking for a lot of customizations, give the vendor a reasonable amount of time to develop the quote.
• Don't assume that the highest priced application is the best.
• Don't assume that the lowest priced application will, in the end, have the lowest cost of ownership.
• Include key stakeholders in the decision. They might slow down your process, but they might provide input or come up with questions that you didn't consider.
• Conduct a thorough trial of the application before you make your decision. 

The task of narrowing down your short list should definitely take more than a couple hours. But it should take less than two or three weeks. If this task drags on taking months and months, you're probably ready to develop the short list. You're probably using your research and evaluation to further refine your requirements and the features you need. And that's okay. It's just better to try and manage expectations internally with your colleagues, and externally with any vendors you are in discussions with.
As you are the gatekeeper who determines which vendors make the short list, everything always comes back to your initial searches on Google. If you are getting stuck bigtime, remember to mix up your research and searches. Make sure that you drill deeper beyond the generic single-word searches, but instead learning how to do more sophisticated so-called "long-tail" searches. Then Google can start become your friend, not something that just boggles your mind.

With an uneven questionable short list, decision makers can get spooked, the true benefits of the initiative will not be understood, and the project can be derailed.  

However, with an excellent well-vetted short list, your colleagues will thank you after the demo meetings and trials because of the high-quality vendors you have chosen. 

Pacific Timesheet is a leading time, expense, asset and log tracking software vendor that is ready to work with you if you think we can meet your most important business requirements and needs. Practicing what we preach, we work very hard to serve customers using the vendor best practices outlined in this article. Our goal is to make it easier for you to evaluate us. After you have developed your short list, and if we make it, we would love to help you test our applications with thorough and active free trial evaluations to see if we are great fit for your company.