One of the many things that excites people about what we do here at OpsClarity has to do with the DNA of our fantastic technical team. We stacked the deck with veterans in data science, large-scale systems engineering and data visualization to create the platform that Ops wants and needs to operate in today’s hyper-scale, hyper-change real-time technology world.
We talk about our team of experts who collectively hold nearly 50 patents in data science, machine learning and data visualization, but we figured you might actually want to meet the men and women behind the curtain. So we are kicking off a little Q & A session with a new OpsClarity team member each month to give you a chance to get to know a little bit more about them.
My first victi… I mean, my first honored interviewee, is Abhinav Vora on the Analytics Team. Hope you enjoy the interview.
Jana: Hi Abhinav, I know you are really busy, so thanks for sitting down with me. I know I see you almost every day during the week, but what I want to do first is learn more about you and what you like to do outside of the office. So, first question. What do you like to do outside of work?
Abhinav: OK. Well, I like to do some types of fitness related stuff. I play squash and like running. Besides that, I love to read.
Jana: Nice. Reading, ok. What was the last book you read?
Abhinav: Hmmm. Well, the one I am currently reading is called “The Master Algorithm.” It’s about AI.
J: Ohh Awesome. Is it fiction or is it educational? It sounds educational, but you never know with some fiction titles these days.
A: It is actually not fiction. Although the topic is something that we tend to relate to science fiction, Artificial Intelligence. This is a book that was written by an academic, but is written for a general audience. It talks about what the path is to artificial general intelligence, and how there might be one master algorithm that the human brain uses to learn everything. So, it definitely not fiction. Although the applications may sound like fiction, the technology is becoming more of a reality every day.
J: Very cool, but do you think that it might be the path to Skynet? Ha!
A: [Laughs] I don’t know, but it is very interesting. You know one skeptical point of view could be that AI has always been five years away, but I think that a practical measure of AI may be “software that does stuff we consider impossible … kinda like magic,” and so getting to that next big step of what we consider impossible today may be 5 years away. But true general intelligence may be a while away.
J: That is really interesting stuff, and I know we love talking about AI here. So, what did you do before OpsClarity?
A: Before OpsClarity, I was working on search for a while. I worked on search at Facebook. People search primarily. I also worked on Web search and Web ads before that at Bing. My primary job has been working at the intersection of systems and ranking/relevance and application of machine learning.
J: Are there any projects or milestones that stand out to you that you have been really proud to be a part of in your career?
A: One of the things I feel like I am really proud of is my work on Bing ads. I was one of the founding members of an ad relevance team where we built the final relevance and click prediction system. It’s always fun to build a system from scratch. I could really say the same thing about OpsClarity as well, but you know, I think that I will have a much better story looking back after a few years.
J: Speaking of OpsClarity, Would you tell me a little more about what you do at OpsClarity and what you work on?
A: Sure. I have worked on a few different things over the past 20 months or so. I have worked primarily on the topology discovery. This application detects what applications are running, what are the clusters, what the connections are between those clusters and all that type of stuff. I also worked on some other streaming systems before that, primarily our data aggregation pipeline. I also help Alan, our VP of Engineering, out with leading the data analytics track – which includes other things such as anomaly detection, health, etc. – and planning the work, sprints, etc.
J: What would you say you enjoy the most about working at OpsClarity? Aside from super awesome co-workers like myself, that is.
A: Well, [laughs] I think that pretty much sums it up. The team is great, and a distant second to the team is the work. In the sense that, in a startup you have a lot of freedom to start from scratch and not have to work from legacy. I think that is a huge strength in any startup and we have a lot of that at OpsClarity. I enjoy that the most.
J: Building from a clean slate is pretty fun. What do you think is the most challenging thing you work on here?
A: I think in general everything you work on is real-time. And because it is real-time, we need to break new ground with how streaming systems are built and engineered. I think that is the single most challenging part of OpsClarity. It is something that is relevant for pretty much all the analytics and data pipelines that we have. It is a fundamentally different paradigm than the batch-processing world. That to me is one of the most differentiating things about us, but also the most challenging.
J: On that note, what do you think it is about real time data processing that makes it so difficult for people to keep up with?
A: I guess it is a relatively newer paradigm and requires streaming algorithms as opposed to working on a fixed size of data. For instance, if I were to take 100 data points, and determine the average of those data points, it is simple to compute. But, if we are going to take 100 data points, and compute the average, but right now I only have 20 data points, it is exactly the same problem but more difficult to compute because sometimes data points get lost, sometimes order matters, sometimes it does not. Those things require us to redesign the same algorithms to operate in a different way. That is what makes it the most interesting and the most challenging.
J: We definitely live in hyper-change world these days that adds a lot of complexity. So, last question – I would love to get your recommendations on what kinds of educational events/conference or meetup you think are considered can’t miss for someone looking to be on the cutting edge on these real-time data processing problems?
A: Well, there are a couple I am personally familiar with. I would say Strata Hadoop, is certainly very relevant, and the other one is Velocity. Another is the AWS re:Invent show, which is more general but tends to cover a lot of these topics just more broadly. Those would be my big three.
J: All great suggestions. And, OpsClarity will be at Strata, so maybe you will be a big internet celebrity after this interview and people can stop by our booth there and meet you in person when you aren’t off soaking in all you can at the sessions there. Thank you so much for sitting down with me today, I hope it wasn’t too painful for you.
A: No, it was fun. Thanks!
Much gratitude to Abhinav for sitting down with me, for what is hopefully my first interview in a series of many to come.