Technology-Assisted Review: Four Key Questions

Guest Blog by Joe Garber, Vice President of Marketing at RenewData

There’s no question that Technology-Assisted Review is a hot topic in eDiscovery circles right now. A quick Google search certainly confirms that premise, and reinforces that organizations are increasingly looking for defensible, cost-saving measures to apply to the most expensive aspect of eDiscovery.  However, what’s equally evident is that there isn’t a commonly accepted understanding of what this term actually means and, as a result, the need for market education is clear and widespread.  Over the summer, my team and I have traveled from city to city across the U.S. to discuss these important issues with industry peers.  During these highly interactive sessions, we routinely find ourselves addressing a handful of questions.  I have identified four of the most frequent questions we are asked, as well as the “consensus conclusion” achieved among these groups.

What is Technology-Assisted Review and Why Should I Care?

Consensus conclusion: Technology-Assisted Review was borne out of organizations’ desires to control cost in the portion of eDiscovery (review) that generally accounts for roughly 75% of their total spend.  It is unlike traditional linear review that is highly manual and involves the interplay of humans and computers – often overlaying a variety of technological approaches such as keyword search, clustering, relevance ranking, and sampling – to vastly expedite the review process.  Technology-Assisted Review has been proven to save up to 80% of total cost versus linear review, which can add up to millions of dollars for even a single matter.

Are all Technology-Assisted Review Solutions the Same?

Consensus conclusion:  No.  Today, there are two broad categories of Technology-Assisted Review – one that leverages artificial intelligence and another that relies on a human’s understanding of language to identify potentially relevant data in a document collection.  The artificial intelligence-based approach provides quick insight into the matter and may require less oversight from senior attorneys, but there can be a “blind spot” in this process.  A few years ago, the common practice was to review as few as 500 documents as a “seed set” in order to train the system on what to look for within the collection. But with data volumes increasing and better education on semantic patterns, a best practice is now to build a seed set of approximately 10,000 documents.  Alternatively, the language-based approach makes document coding decisions based on the specific language contained within each document.  This process is easier to understand and explain to all parties than its artificial intelligence cousin (can you explain the inner-workings of Latent Semantic Indexing?), provides more transparency into the coding decisions and makes it easier to audit reviewers in real time, and creates a reusable work product that can provide even greater efficiencies in the future.

Does Case Law Support the Use of Technology-Assisted Review?

Consensus conclusion:  Case law is quickly emerging to support the use of both categories of Technology-Assisted Review.  Two specific cases, from highly respected districts, are particularly notable: Judge Peck’s February 24^th order in Da Silva Moore v. Publicis Groupe & MSL Group,No. 11 Civ. 1279 (ALC) (AJP)(S.D.N.Y. Feb. 24, 2012), and Kleen Products v. Packaging Corporation of America, Case No. 10 C 5711 (N.D. Ill. April 8, 2011). In Da Silva, Judge Peck specifically holds that “(Technology)-assisted review is an acceptable way to search for relevant ESI in appropriate cases.”  While Judge Peck comments on a matter that involves the artificial intelligence approach in this particular case, the general principles he highlights – leveraging technology to expedite review, focusing on quality, and sampling to ensure reasonable results – support both approaches.  In Kleen, Judge Nolan held for the producing party’s use of a language-based approach for a number of reasons but specifically because their approach has been embraced by the court system for years.

How Do I Choose the Right Alternative?

Consensus conclusion:  It depends.  There is a whole spectrum of review acceleration solutions available in the market, and choosing the right one (or often a combination of them) depends on your company’s litigation profile, your data set, and the time, cost and risk sensitivities of each unique matter.  The artificial intelligence-based approach to Technology-Assisted Review often has two elements that are in play: the need to arrive at quick decisions early on in the litigation, and enough time is available to read up to 10,000 documents for a seed set.  The language-based approach is most appealing when transparency and reviewer auditability are of paramount concern, and when an organization wants to incorporate this approach as a regular business practice.  Be cautious of tying your success to a single technology platform because each matter is unique and each may require a slightly different methodology to achieve optimal results.  If you need help on compiling the right solution, take the time to find an expert, because the cost and risk of making a mistake in eDiscovery can be severe.

To learn more about review acceleration, and the two key alternatives to Technology-Assisted Review, an excellent white paper written by Enterprise Strategy Group is available here.

About the Author

Joe Garber is Vice President of Marketing for RenewData.  During his 18-year career, he has served as Director of Market Strategy for Autonomy (an HP company), a management consultant for IBM, led marketing and product management for a variety of successful technology startups, and served as a press secretary for a U.S. Senator.  He holds a Bachelor of Arts degree from Pepperdine University and a Master’s of Business Administration (MBA) from Cornell University, where he received the prestigious Park Leadership Fellow award for “demonstrated leadership and academic excellence.”

What Does Your Language Say About You?

What comes out of your mouth always makes a difference.

Posted by Adam Weissman, Director of IT and Legal Technology at Glenmont Group.

No matter how small or big the organization, have no doubt that the messages communicated from anyone and everyone who holds a management or leadership role will guide the culture of the company. In the last decade alone, news stories have documented the wildest success stories and catastrophic failures of some of the most recognized corporate names in the world. When you read or listen to interviews with current and former employees, it is seemingly common for these recounts to point directly and specifically at the messages being delivered from the top-down. References to emails, company-wide memos, and management directives that enabled certain behaviors and fostered particular protocols are brought to light. They refer to singular management-delivered and -driven messages that inspired an entire organization to do better or, conversely, connect specific incidents and communications that ultimately and negatively altered the undercurrent of employee sentiment.

In a recent article entitled “How Language Shapes Your Organization” by Kevin Allen, a contributing author to Harvard Business Review, the impact and power of the words used by managers and executives is analyzed. His commentary assigns responsibility and accountability to even the most casual of communications from those in a leadership function.

http://blogs.hbr.org/cs/2012/07/how_language_shapes_your_organization.html

Hocus Focus

Posted by Susan Beck, Project Coordinator at Glenmont Group

After reading Adam Weissman’s Blog from yesterday entitled, “Clowning Around Is Serious Business”, I wanted to take a moment and focus on an additional way to stay narrowed in on the task(s) at hand. Adam referred to an article that touched on taking breaks in order to keep your mind focused on your work. While his blog touched on the breaks in your workday, I would like to touch on the work that you do within your workday.

Multi-tasking is a difficult thing to do. Although you think you are multi-tasking, the brain seems to only truly focus its energy on one task at a time. I notice this most when I am reading. I’ve been reading a lot recently. While relaxing and kicking back with a good book, I often find myself reading a passage that will trigger a memory. Upon thinking of the memory, my eyes will continue to read the passage, floating through the words as if I am actually reading them. Although, when I finish my thought and look back at what I just read, I remember none of it. Work can be very much the same. The link to the article below uses an example of talking on the phone while sending an email. Your mind focuses on one, and then the other, and then back to the first. There is no way to have your brain listen to, and absorb, what your caller is saying while instructing your fingers to move to the right keys on the keyboard. Try it. You may think it works, but your brain is actually rapidly switching between the two tasks.

In multi-tasking, your mind is simply switching modes and allowing you to handle your responsibilities inefficiently. If you took the time to give each task/account/issue private one-on-one time with your brain, you would surely notice a couple things. 1) It may not take as long as you expected to finish each item, 2) The completed task will have been done more completely and detailed than you normally would have done, and 3) Your mental processes will seem far less scatterbrained than previously. By cutting out interruptions to your day, you will find that productivity increases and your day runs much smoother.
This article, written by Karen Sladick of Organize 4 Results explains the mind-set involved in this change. http://bit.ly/MxDtA2

Clowning Around Is Serious Business

Posted by Adam Weissman, Director, IT and Legal Technology at Glenmont Group

There’s an old saying, “Time flies when you’re having fun.” I have often found myself in the middle of a phone call or pausing while writing an email and peeking down at the clock, only to realize I haven’t moved from my desk since I sat down at it 5 or 6 hours ago. I have fun in my job, so I suppose the saying applies here. A large part of recruiting is about getting into a rhythm and trying to maintain it. While most recruiters strive to find a steady groove during their day, it is equally important to remember to take a break once in a while. Repetition can lead to great conversations with candidates and clients, which can turn into confidence, which hopefully leads to success and a sense of accomplishment. However, it can also be counter-productive if you don’t flick the mental light switch off for a few minutes, let the bulb cool once in a while.

Susan Adams, from Forbes.com, has some suggestions for the types of activities some professionals utilize to keep themselves sharp and focused throughout the work day. Of course, every office is different, so you may find some of these are more applicable to your circumstances than others, but you don’t have to be a recruiter to benefit from taking a break from time to time.

http://www.forbes.com/sites/susanadams/2012/06/18/eight-ways-goofing-off-can-make-you-more-productive/

So, This One Time, At Band Camp…

Posted by Adam Weissman, Director, IT and Legal Technology at Glenmont Group

I am always intrigued, often confused, and sometimes impressed, by the methods by which people prepare themselves for job interviews. A big discussion topic in the employment world is, “How do I distinguish myself as a candidate for a job?” I have learned over the course of my life as a recruiter that assuming an individual’s innate ability to interview well (i.e. dress appropriately, make eye contact, articulate ideas and information in a detailed-but-concise manner, ask questions that are both relevant and appropriate for the circumstances and the specific job or company, etc.) without guidance, ends poorly more often than not. Every time a candidate (whom we represent) interviews with one of our clients for an open, viable position, the outcome is a direct reflection on both the individual recruiter and Glenmont Group as a Search Firm, good, bad or indifferent. Each interview conversation, for each job, with each organization, can and must be tailored in order to convey the right message and fit.

Regardless of the level of position we are recruiting for, any candidate who interviews with one of our clients, has been prepared for their interview by a Glenmont Group recruiter. We are not providing canned responses, confidential information, or question lists, as we find many interviewers want to elicit different information from one interview to the next. Clients sometimes even base their inquiries within the context of the conversation with that specific individual. Rather, we focus on counseling these talented professionals in their mindsets and overall approach to the conversations and meetings they are about to have – the exchange of information that, for some, could have a profound impact on their professional and personal lives.

Ahh, the ever-elusive, no-fail tactics to ace every interview do not lie here. However, I read an article this morning by Vickie Elmer on AOL Jobs that identifies one interviewing strategy that can be beneficial to prospective job applicants, particularly in the professional setting. Elmer encourages interviewees to give thoughtful consideration to personal stories that can provide potential employers a highly-relevant, valuable insight into the broader makeup and character of that individual.

http://jobs.aol.com/articles/2012/05/16/how-to-tell-your-story-in-a-job-interview/

Ediscovery New Zealand Style

Posted by Kate Potters, COO, Glenmont Group

As I’m preparing to go on vacation to visit my daughter in New Zealand, eDiscovery is not on the top of my mind. However, discovery is a global issue and New Zealand is no different. Although less litigious than the U.S. or Europe, this island country implemented new discovery rules in February in direct response to the rising cost of discovery dealing with large volumes of ESI. Sound familiar?

More fascinating than the EDRM model and the technical aspects are the stories behind the cases themselves. Clearwell’s eDiscovery blog mentions a case in New Zealand involving an internet billionaire, Mr. Dotcom. That’s right – he officially changed his name. He’s involved in the largest copyright infringement case in history. The U.S. is looking to extradite him because his business allegedly allows users to illegally download pirated material. His website has been “seized pursuant to an order issued by a U.S. District Court.” Here is the story of his arrest at his mansion outside of Auckland, http://www.dailymail.co.uk/news/article-2089954/Megaupload-founder-Kim-Dotcom-sprang-electronic-locks-Bond-villain-lair-police-swooped.html

He was released on bail and waiting a hearing in August on his extradition to the U.S.

http://www.clearwellsystems.com/e-discovery-blog/2012/03/29/ediscovery-down-under-new-zealand-and-australia-are-not-as-different-as-they-sound-mate/

If you’re interested in learning more about eDiscovery in Oceania read The NZ E-Discovery Blog http://www.e-discovery.co.nz/blog/a-judicial-commentary-on-new-zealands-new-discovery-rules.html or http://www.e-discovery.co.nz/

I plan to forget about business for a while and have some fun, but if I happen to learn anything new, I’ll be sure to post a follow up!

Did Your College Do Its Job to Get You a Job?

Posted by Adam Weissman, Director, IT and Legal Technology at Glenmont Group

As a recruiter who specializes in legal technology staffing, and as someone who has also been on the employment search side, I constantly keep a close eye on the emerging trends that affect hiring and impact various industries and job markets across the country. I regularly communicate this information to my candidates and use it to counsel and educate my clients. One of the uglier trends in America right now is, due to the downturn in the economy and instability of the job market, many families are forced to make tough decisions when faced with the steadily increasing costs of pursuing higher education. A topic that has been hotly debated for decades in Corporate America is whether obtaining a four-year college degree really gives individuals a so-called “leg up” getting that first job over those who either have a 2-year degree or no degree at all. As families decide how to afford to give their children that academic advantage for their future, they should examine what they are truly investing in. Here’s some food for thought: If you graduate from a four-year school (we’re talking non-trade-specific institutions) and are not pursuing post-graduate studies, are you assured you have been equipped with the necessary skills and knowledge to be considered a good hire for companies hiring in your field of study? The better questions are: For all the money it costs to attend a four-year college or university, do you feel your school has truly invested in establishing majors and curriculum tracks that stay current with the employment trends? And, is it their responsibility to do so?

Please participate in the attached poll below…your input is incredibly valuable, and the results will be published within the next 2 weeks.

In addition, I recently read an article on Yahoo! News discussing the rise in recent college graduates struggling to find a job or who are grossly under-employed. The perspectives shared demonstrate the widening gap and disconnect between what skills, experience, and knowledge businesses in the U.S. look for in potential candidates, and where they turn to find those individuals (eh-hem, future blog).

http://news.yahoo.com/1-2-graduates-jobless-underemployed-140300522.html

Optimizing Your Social Media Presence

Posted by Susan Beck, Project Coordinator at Glenmont Group

I came across an article from about a month ago through Legal Technology News. It was written by Kevin Nicols of KLN in San Francisco and touches on 8 different ways to achieve your professional goals through social media. This includes Facebook, LinkedIn, Twitter, and a number of other avenues. Although the article is geared toward lawyers, associates and the like, I find the social media processes Kevin comments on to be relevant for anyone. Whether you are in the legal industry, working for a technology company, or handling the marketing for your company, you can adapt these skills to your social media presence to help build your professional appearance.

Here is a link to the article, followed by a brief overview: http://bit.ly/w8Hzw0

Are you looking for a good way to promote yourself? Use these 8 tips to successfully maneuver the whirlpool that is social media.

1.  Create a “brand” for yourself: Utilizing your current position and your lifestyle, use pictures and words that accurately describe yourself. Don’t be someone you’re not.

2. Use Twitter for marketing: Just make sure not to do something that will damage your “brand”.

3. Search yourself: Be sure to see how the world sees you. Google yourself every now and again.

4. Check your network: Are you only reaching your immediate friends from your high school? Does your message travel across the globe? Take a look and see who is getting your message – is there a better way to promote yourself?

5. Target markets: Search Engine Optimization will allow your message to reach your target market first and foremost when you utilize new content often.

6. Status updates: How do you best utilize status updates? Is there an easier way to tackle this task?

7. Sharing knowledge: Be sure to find a good place to share messages. Link this to your social media profiles.

8. Communication is key: Be sure to keep an open line of communication with your audience.

Five Tips On How To Work With Recruiters

Posted by Dana Fink, Director of Staffing at Glenmont group

After a crazy week of both candiates and clients not being completely forthcoming with all information needed (fibbing is putting it mildly) I felt a back to basics lesson on how to work with recruiters was required reading. The need to be completly forthcoming and honest is the only way to manage your career search.

This simple article on recruiter.com was just a quick “how to” I thought would be helpful.

FIVE TIPS ON HOW TO WORK WITH RECRUITERS

Working with recruiters is the next best thing to working with hiring managers. Recruitment professionals know their clients and they pride themselves on knowing their candidates in the hopes of making the right connections. In order to make this happen there are a few things you need to know:

TELL THEM EVERYTHING: When building a relationship with a recruiter, this is no time to be shy. Recruiters need to know everything there is to know about you and what you are looking for in your career, i.e. salary, perks, specific employers, etc. Building an effective relationship with a professional recruiter starts with trust and honesty, so be forthcoming with your requests.

FIND OUT EVERYTHING: Relationships are a two way street. When working with a professional recruiter you need to know as much about them as it relates to their capabilities and ability to help you secure employment. Do a thorough background check on the recruiter and/or the company. Review all the social networking sites where the recruiter does business and if they do not recruit via these new mediums, be aware; it might mean that they are not keeping up with workforce trends and new recruiting practices.

COMMUNICATE OFTEN: Unfortunately securing a recruitment professional does not mean you will secure employment immediately. There can be a lot of waiting involved. Sitting still and waiting for the phone to ring is not an option for you. You are part of the communication equation and you must take an active role in all aspects of your career.

Approach your job hunt like you already have a job. If you see something online, hear about it on a website, see it on the news – contact your recruitment professional and collaborate with them on a game plan. Good recruiters are savvy and well connected, like real estate agents. They tend to know where all the hot properties are. However, they can not be everywhere, so the more hands they can have working in your favor the better.

BE REALISTIC: Understand that you are not the recruiters’ only client. Schedule standing meetings no matter how short; just get them on the calendar. Recruiters understand your urgency however they have other clients asking for the same things. Work with your recruitment professional to set realistic goals that are effective and actionable.

Please Don’t Tell Me About Yourself — Behavioral Interviews

Posted By Dana Fink, Director of Staffing at Glenmont Group

Managers have a very important job in today’s business environment. Interviewing and selecting the right personnel are two of the most critical things they must do to reach organizational goals. The hiring decisions made today will influence their organizationfor years. Developing a profile of the behaviors you want your employees to exhibit is a critical component of the selection process. Many managers utilize a behaviorial interview as a way to measure future actions employees will make If hired. This behavioral-based interviewing program was developed by Dr. Paul C.Green, president of the consulting firm Behavioral Technology, located in Memphis. Dr. Green created this selection process through his human resources experiences with a broad range of organizations. The approach reflects his strong belief that scientific research can be combined with practical experience to provide a meaningful way to conduct employee selection.

 The article below gives a short synopsis of this meaningfull hiring tool

By: Mark Murphy, author of Hiring for Attitude (McGraw-Hill, 2011)

A seismic shift from skill to attitude has taken place in the hiring world. Technical proficiency, once a guarantee of lifetime employment, has become commoditized in today’s job market. Between the global labor market and the high unemployment of the Great Recession, there’s a large supply of technically-qualified candidates for almost every available job.

Job skills are necessary, but they’re not sufficient.  It’s attitude — not skill — that is the best predictor of new hire success. In a Leadership IQ study of 20,000 new hires over a three-year period, 46% failed within their first 18 months.  And 89% of the time they failed because of attitudinal reasons; technical skills barely made the list.

Hiring for Attitude How are the best companies hiring for attitude? First, savvy leaders identify the specific attitudes that create success in their unique cultures and environments.

We call these key attitudes “Brown Shorts” — a strange name that pays homage to Southwest Airlines and their culture of fun.

Brown Shorts draws from a story I heard from a former Southwest executive about a round of hiring for new pilots (typically serious folks dressed formally). The Southwest interviewer invited this serious bunch to get comfortable in a pair of Bermuda shorts (brown in our story).

The shorts were part of the Southwest summer uniform, but it was an invitation that seemed too ridiculous for many of the pilots who immediately declined the shorts. And that told Southwest that these folks may be great pilots, but they just weren’t going to fit a fun-loving culture.

Now, just because you put on the shorts was no guarantee of a job, but it was a good indication that you just might fit their fun attitude.

Your Brown Shorts probably won’t to be “fun.” But they will be a list of the key attitudes that define your best people and your worst people. And when you model your interview questions around those Brown Shorts, you’ll discover who will (and won’t) succeed in your organization.

To hire for attitude, you’ve also got to interview for attitude. And that means ditching interview questions that don’t help you assess candidates’ attitude. For instance, the best interviewers no longer ask:

• Tell me about yourself.
• What are your strengths?
• What are your weaknesses?

These questions are too vague, inviting canned answers that don’t reveal attitude. (If every candidate gives the same answers to a question, it’s not worthwhile asking).

Behavioral Interview Questions So-called behavioral interview qustions  are also often ineffective for assessing attitude. Yes, asking about past behavior can work, but most behavioral questions contain a “tip off” that tells candidates how to give you the “right” answer.

The main issue with behavioral questions is that they rob you of your chance to find out if someone is a ‘problem bringer’ or a ‘problem solver.’ Let’s say you ask a candidate a pretty typical behavioral question: “Tell me about a time when you had to adapt to a difficult situation.”

This question may sound fine, but the word “adapt” ruins it. That single word signals that you only want to hear about a time the candidate “adapted” (instead of the hundreds of times they failed to adapt).

In the case of true high-performer candidates, these folks have plenty of examples to share that describe a time when they successfully ‘adapted’ to a difficult situation.

If you asked them about a time they “faced” a difficult situation, they’re naturally going tell you not only about the difficult situation, but also about how they adapted to it. For high performers, it’s practically impossible to even imagine ‘facing’ a difficult situation without also successfully ‘adapting’ to it.

But for problem bringers (low performers), the word ‘adapted’ renders this interview question ineffective. Problem bringers have faced countless difficult situations. But it’s unlikely they’ve successfully adapted to any of them.

In fact, the times they successfully ‘adapted’ probably constitute such a tiny fraction of the times they ‘faced’ difficult situations that it wouldn’t even occur to them to search their mental database and find an instance where it happened. And that’s something you want to know about.

But when you introduce a leading interview question, you’re not giving them the chance to disclose that information.

Just remember that attitude is the key driver of new hire success. And if you’re willing to rethink your hiring process to discover if your candidates have the right attitude, your hiring success will skyrocket.