The Beginning
On-Campus Recruitment
The day started with a pre-placement talk during which Coviam gave a glimpse of their products, work culture, and experiences of being a part of Coviam. The session was crisp yet informative.
Interview Session Ambiance
The interviewers from Coviam were highly knowledgeable and friendly. They created a comfortable interview session for the candidates and brainstormed ideas, suggestions, and solutions whenever required. When the room environment became tense, they cracked jokes to spread positive vibes and break the tension.
Optimistic Environment
The objective of the interviews was to test the core knowledge of the candidate. Ample time was provided for each question that was asked. Being nervous during the interview felt normal but the interviewers created a positive atmosphere where we could think and confidently answer the questions.
The Training
The Sumptuous Welcoming
Coviam team welcomed the selected candidates with a MacBook Pro and company bags and other accessories. We were introduced to the CEO, the HR team members, and other leads. The structure of the company and working strategies were broken down for us. They filled us with advice on how to kick-start our careers in the best possible way. We were informed about the various projects that were undertaken by the company in the past years as well as the ongoing ones.
Training Highlights
The training started from the next day where we were introduced to the VP-Engineering, Phani Kumar Patchava. He trained us thoroughly from the basics to the advanced level. The training was aimed at individual betterment. It was challenging yet fun to learn on a daily basis. The assigned tasks helped us get better in our technical knowledge as well as help develop a professional thinking process.
Commencing Of Projects
After training we were handed down projects. The first one was about building an e-commerce site. The project was assigned to our group. It was quite a learning experience as we had to distribute the subtasks among ourselves and complete them. It improved our teamwork skills teamwork and gave a boost to our confidence as we completed it successfully and well-presented it. The next project was building a question and answer (Q&A) bot, which was equally challenging. We were given two days for generating ideas for this research project. We brainstormed design ideas and planned its implementation. Through the course of time, one thing that we learned was that planning and designing the framework was of utmost importance in this field. After the successful implementation of this, we presented the project to the leads and took care to improve on the shortcomings from our first presentation. We got better with each project as we improved our thinking approach, ideation, planning capabilities, teamwork, presentation skills and a lot more.
The Formidable Yet Inspirational Journey
The time had finally come when all the training was put to test. I was assigned to work with the Data Analytics team where my lead was and still is, Anwesh Roy and mentor Gokul Palanisamy.
To Begin With…
I’ll be lying if I said that I was fully confident. The task at hand, which was on the surface, seemed too challenging to take up. When I got a customer requirement, this project demanded immense planning and involved the use of many new technologies, which I had not heard about. We were given two days to plan for this and develop an algorithm and implement it.
I had self-doubts on whether I might be able to complete this project but with constant motivation and ample help and tips from my mentor and lead, I successfully completed it. This was a necessary boost to my self-confidence and a sense of accomplishment kicked in. The learning curve has been so exciting to ride on.
Problem Statement
This project involved retrieval of tabular customer unstructured data from a pdf document and converting it into an excel sheet with the same alignment.
Solution Analysis
For parsing the document, only a paid tool was available but then after figuring out the internal working of this, I was successfully able to create a parser for the document. Thereby, it removed the dependency on the paid tool.
I tried various algorithms, such as grid-based algorithm, pattern-matching algorithm but didn’t achieve enough accuracy. So, I switched to coordinate based algorithm by parsing the document into a required format that could also solve the alignment issues that I was facing. This algorithm gave a better accuracy rate. This was my first experience in algorithm design.
New Challenge
The next project involved working on a research-based project, resume parser, to crack the problems, issues and increase the accuracy. This had to be undertaken step by step.
Accomplishing Successful Implementation
The implementation of the project was successful but to be frank, it did not turn out to be easy. As I was working on a research-based project, I had to fully analyse the problem. Analysing it involved tracing the working of the algorithms and finding out issues in it. I studied, understood and rectified all the internal working of the project, the tech-stack such as Named-entity recognition, ElasticSearch, Redis, MongoDB, and Apache Tika. The process of rectification involved dire hours of trial and error and finally, if it demanded more, we came up with alternate solutions through hours of debugging and brainstorming. I discussed alternate solutions with my mentor, who guided me through the process. He always welcomed ideas from my side. Each solution was tested to find out the accuracy and outputs. The solutions were compared with each other to find the best fit for the purpose. Once an issue was correctly solved and done, I moved on to the next issue. The process continued until the final code was up and running, free of issues and bugs and perfectly accurate.
The Fruitful Meetings
I had meetings at different stages where my lead and mentor sat together with me, where I discussed the progress of my work and also addressed the issues that I was facing in implementing the tasks in hand. The meetings were not just general gatherings, they were complete brainstorming sessions where I had to put forward our progress and also discuss the problems that I was facing. Our mentor and lead were patient enough to listen and also understand my problems to provide quality solutions and guidance to march forward. The meeting duration was never fixed – it lasted until each one of our doubts and problems were put forward and solved. The meetings have ended up being a huge platform for learning and analysing the mistakes and our own shortcomings.
How I Went About Challenges
Problem Statement 1
The major issue in this project was with the parser i.e Apache Tika parser as it produced raw text in an unstructured format. Eg: Jumbled content, Top Content came at the Bottom, Bottom content came at Top and Garbage raw text, Unnecessary raw text, Tabular content were converted into mixed data with column-wise alignment and sometimes row-wise alignment.
Solution Analysis
To correct this, I checked various plugins with the same parser but nothing worked out accurately. Then I learned about the internal working of the parser. Finally, with certain manipulations within the internal working component of the parser, I had achieved the right output.
Problem Statement 2
The next big issue was – How to extract educational qualifications from a resume. This was a major issue as different resumes used different formats, headings, and style to describe educational qualifications. Also, the positioning of this section in the resumes was different. For example, a ‘B.Tech’ degree can be written as ‘B.E.’ or ‘Bachelor in Technology’ or ‘Bachelor in Engineering’ or ‘Graduation’ or simply ‘Engineering’. Retrieving this from the resume was the tricky part. Also, the order in which the candidate writes the degree, college, percentage, year of graduation and score also varies to a huge extent.
Solution Analysis
I at first struggled to retrieve the heading but after analysing that it was not so accurate, I planned for extracting the educational section. It again failed with different cases. Finally after hours of dire analysis and designing of algorithm, I finally came up with an algorithm design using Named-entity recognition that can now retrieve educational qualification from anywhere in the resume, that too in any format.
Problem Statement 3
To extract the work details of each candidate from the resume was the next big challenge as these too were in different formats and varied in nature. For example, employer details followed by start year and end year; at the same time it followed duration and at the same time it followed project details and all these parameter orders can be jumbled. However, this was not only the problem. The problem lay in company name as well. Eg: IBM could be written as I.B.M or even by using the full name as “International Business Machines” and same goes for date and duration format.
Solution Analysis
At first, I thought the same algorithm used for education could be used for this as well but that did not work. I had to design a whole new algorithm that could search the work details from the resume and retrieve it successfully. Also, different amounts of resume were fed in as input to find the accuracy rate of the designed algorithm. Retrieval of technical details from the resume also followed as a big challenge as the details were written in different formats.
Problem Statement 4
To extract the contact information, retrieving the contact number and email id was relatively easier than the name as these have a fixed pattern in them. The big question that posed in front of me was how to retrieve the name from the resume. Eg: The name can be at the top or bottom and with different format. Eg: Anshuman Varshney can be written as Name: Anshuman Varshney as well.
Solution Analysis
To retrieve the name, I analysed many resumes to identify how people are writing the name and tried applying similar algorithms as before. With extreme analysis I figured out the right algorithm design to extract the information successfully. The real challenge is that no matter how the resume presents us the information, my algorithms have to rightfully extract and present the information. Dedicating so many hours and designing algorithms independently has served as a boost to my ability and confidence.
At this stage, the resume parser was deployed into production and I had started getting more projects. The resume parser yielded in giving us really good response.
New Challenge
This project involved extraction of all the questions and answers from the document(ie. pdf, docx, doc, txt, rtf etc.) or any website URL.
Solution Analysis
This was a challenging task and the compatible design had to be developed. The task was challenging but I had enough confidence. I had much more professional thinking and skills than I had at the time of joining the company. I explored new things and also took projects and developed them as they came.
The documents and the URL posed a challenge of understanding the presence of a question and an answer in the document. There were tricky challenges involved but I was successfully able to complete the task.
In order to retrieve them, different designs were required and certain intelligence algorithms had to be placed in them to achieve the tasks. I was more confident this time and it greatly helped me in completing the tasks in hand and gather good responses for this project.
The handling of different projects helped me develop technical skills. The successful deployment of my projects enabled me in gathering enough confidence.
The Buildout & Transformation
The Incomparable Mentors
I am deeply privileged to have one of the most impressive mentors and an irreplaceable lead, both of who are always available to address any kinds of doubts that I have. They are very understanding and always appreciate any enthusiasm I show at work. They have always encouraged me to work and think independently and are highly appreciative of the solutions and work that I present to them. My lead always insists that it is not the amount of work that we have completed that counts but the amount of work that we have delivered with complete quality. They provided me with quality guidance and have helped me create an attitude to understand any problem by breaking it down right to its basics and made me realise that the only way to learn is by dedicating our sincere self and time.
Adaptation To Be Better
Within a couple months, the learning process has been really impactful. I transformed from a graduate to a professional. I was taught to challenge myself, take up challenging projects and work and code to see the successful implementation of the same. My mentor guided me to a whole new level of self-confidence and thought process. I think of solutions and try and implement it and to see them being executed successfully in the real world. To think like a professional and to be independent at work is something that has been achieved with total grace. My days of self-doubt and being dependent has been replaced with a curious and thirsty mind that is willing to work my way to perfection.
By Anshuman Varshney
Also published on Medium.
