Enabling and Streamlining global trade across borders for Indian and Overseas SMEs: Connect2India

Connect2India is the one stop global trade integrated platform for 51 million Indian SMEs to effectively trade from India and 40 million internationally competitive overseas SMEs to effectively trade with India.

With 125,000 Unique Business Visitors per month and customers across India and overseas, including large enterprises such as Modi Group, Jindal Group, Rico Industries, Auto Ignition Ltd, South Indian Fertilizers, Kutch Chemicals, Caparo India, Inbisco India, Orion Ropes etc., Connect2India is scaling very aggressively. However, not many know that behind the unique and innovative end-2-end global trade offerings that platform offers to Indian and overseas SMEs underlies a very strong and robust technology engine that is already working with a Billion+ data size, is capable of handling Million+ users, and is making use of Big Data Analytics, Machine Learning and Artificial Intelligence to redefine and evolve the way global trade is being done.

With 200,000+ Google requests per day and sub second response time, the high-scalability, high-availability and high-performance of the platform is pretty self-evident.

A candid Q&A with Pawan Gupta, CEO and Founder, and Ranjan Malakar, CTO on what it took to build the technology platform for Connect2India

How difficult was the beginning, and how did you tackle the initial phases of building your platform?

When we decided to build a platform that will be a one stop source to every global trade need for Indian and overseas SMEs we also knew that we will need to innovate and come up with the best mix of things if we really wanted to make it work.

And so with our tech team of 5 members we started working on the idea. Before starting to develop our PoC, we first tried to understand what we were planning to do, how that can be achieved and to do that what are the initial requirements. Our systematic way of approaching every step in the way is still a core practice in our organization.

We came up with the following requirements for our MVP:

• 1 million rows of Indian SMEs data - We knew that to solidly pitch ourselves as a technology driven global trade platform the first thing we need was ample of data relating to our field of work, so we set up an initial target to collect relevant SMEs data.
• 5 million rows of global trade stats - We were not in pursuit to simply take advantage of the lack of technology driven trade platforms, we really wanted our product to be helpful to the SMEs, and we identified that one of the major issues which SMEs faced was the lack of data-driven intelligence to understand & analyse the global trading markets, thus, we aimed to collect as much data regarding global trade so that users could make informed decision.
• 5 unique data sources for generating holistic view - little bits of information serves no one the purpose, it can in fact be disastrous at times because people have the tendency to narrow their thinking process to whatever information they found first, if it is bad, they will back off, if it’s good, they will not make any more efforts to know more. Thus we wanted to give our customers a holistic view of things, converting series of discrete snapshots from multiple authenticated data sources into a holistic view.

When was the MVP completed? And was it satisfying to you or did it come with its own challenges? Also, was it scalable enough to act as a full-fledged system?

Our team was determined, as a result we ended up mining more global trade and holistic view data than we had envisioned, however, the real challenge was only getting started, extracting data was okay, it was the application and optimization of that data which was the real challenge. The stateful architecture used in MVP was making server heavy with increase in number of concurrent users thus, we had to move toward stateless architecture to reach high scalability.

Further, Angular being a relatively new technology was not very scalable for google indexing and optimization for huge number of content pages and that required a lot of efforts with millions of pages of ours. Performance was not at optimum level at high loads, as every request had to make a full round trip from browser to web server to application server to data storage and back.

If there was one USP in our minds that we would provide, it was uncompromised performance and usability, yes it took time and trust me lots of efforts, but the satisfaction of the results was more than enough. We faced the challenge right up; made our server completely stateless, built our custom layer of security frameworks on top of spring security for authorization and authentication. We implemented custom caching with our own caching implementation on top of distributed memcached layer and optimized & tuned our DB configurations.

We then went on to modify our schema to store both relational and non-relational data types to save and retrieve the datasets like JSON objects directly from DB; achieving the same results as from NoSQL databases and yet not compromising on database integrity at the same time. We also structured our REST APIs to enable same interface access for both our web app & mobile app and used advanced algorithms to store user data securely by hashing/encrypting user’s sensitive data.

How did all the hard work and innovation paid off? What does the current system look like?

Today, all our efforts help our users get the best global trade platform yet. Based on the efforts in our MVP and resolving the challenges we managed to create a system which;

• The users are able to get global business opportunities by targeting companies based on their trade history, trade activities and buying / selling patterns (derived using regressions, time series, decision trees, k-means and neural network) and are able to close those leads effectively with our global trade intelligence and global trade resources available on the platform.
• The users can identify new markets and trade counterparts for their product and capitalize on global trade opportunities by tracking the real-time price trends and supply-demand shifts across products and countries.
• The users can execute their global trade transactions on the platform using in-built global trade pricing engine, integrated CRM and a personalized global trade control panel with pre-populated trade counterparts, trade analytics and trade resources corresponding to customer’s products.
• The Sentiment analysis using machine learning over trade counterparts’ trade history, holistic view and the scoring engine provides a reliable mean for user to figure out the risk of trading with a trade counterpart.
• Provide advanced global trade analytics and insights using Python’s advanced numeric & scientific libraries such as NumPy, SciPY and PANDAS for data manipulation and analysis. Advanced reports are presented using advanced interactive trade infographics using libraries like d3, nvd3, google charts. This make the platform offer analysis of data using historical trade insights, current trade demands, predictive trade analysis and comparative analysis for various products and countries combination.
• With our efficient custom caching layer implementation with distributed memcache, optimized code & queries for concurrent users and tuned and optimized DB, the system is set for low latency, high performance and increased usage.

 In terms of numbers, how does the platform look like today?

As a result of our team’s continuous efforts and strong emphasis on making technology the back-bone of global trade facilitation, today our transaction size is something like this; 

• Platform data: 1 Billion+
• Global trade stats: 100 Million+ rows
• Listed entities: 5.5 Million+
• Listed entities with product and trade history rows: 35 Million+
• API Response time in single user environment: <200 ms
• Response time in high concurrent user environment: < 800ms
• Number of Google pages indexed: 1.2+ Million
• Number of Google pages in process of indexing: 3.8+ Million
• Platform capability tested for: 1 Million+ users
• Daily rate by Google: 200K+ requests per day
• Unique Organic business visitors: 125K / month

So far, technologically, it has been a journey full of innovation and better use of the available technology, what can more can we expect from Connect2India for the future? 

Connect2India vision is to make global trade so safe and easy with technology such that every single SME could trade globally. We continue to build more and more data sets, derive buyer/seller behaviour, identify buying / selling patterns, build advanced global trade intelligence and augment holistic view with new data sources. 

We expect our technology to make global trade effective and efficient for SMEs by helping them get insights into identifying right products, right markets, right customers, as well as the right price and the right time for trading globally. 

We are looking to get better and better. We are working on getting 1 million unique business visitors per month on the platform as soon as possible to make our platform visible to anyone searching anything related to global trade. To achieve this, we continue to extend the custom framework we have written for AngularJS. We also continue to make our platform extremely user friendly by evolving our UI & UX based on customer feedback. 

Further, we continue to evolve our use of advanced analytics, machine learning algorithms and artificial intelligence technologies to add greater value to our users and are working on several initiatives such as: 

• Comprehensive Global Trade Analytics: We are using advanced machine learning algorithms like K-means Clustering, Decision Tree and Regression Analysis to perform predictive and deductive analytics and make suggestions based on results to determine which importers / exporters should have high propensity to buy/sell what product and at what price at any given point of time. We are also increasing the operational efficiencies by using scoring engine to identify and qualify trade counterparts effectively.  

• Advanced risk management & reputation analytics: with real time Big Data processing to find, match and manage huge number of results, including from news, Google search, social forums, social media, comments and feedbacks to compute the risk of a company. We gather all this information about the company and perform Sentiment Analysis of this information using Advanced NLP and Machine Learning techniques. The plan is also to integrate Fraud detection using advanced ML, IOT and 360 degree VR to validate company’s business premises and facilities.
• Recommendation engine - for both registered users and the site visitors based on their own transaction and activities, activities of millions of other users and the huge trade history, trust score & user ratings and trade patterns data stats. The Recommendation Engine uses advanced Concepts of Big Data Analysis and Machine Learning to recommend any Importer/Exporter about suitable products they might be interested in based on what other importers/exporters similar to them do.

Meet the team which made Connect2India possible

Pawan is Founder and CEO of Connect2India, where he provides strategic direction, business planning, and market & industry leadership. He has successful track record of building business grounds up in diverse sectors including global trade, e-Business products and global consulting.

In his previous role, he led global growth strategy as group CTO for Trilogy & Versata Enterprises. Prior to that, he worked in senior technology roles with Siemens and IBM. Pawan holds a MBA from London Business School and Bachelors of Electronics and Communication from NIT, Kurukshetra.

Ranjan Malakar, Chief Technology Officer: Ranjan is responsible for the core engineering at Connect2India. Ranjan was earlier Principal Engineer at CA Technologies and has led fast paced development from scratch to final technology and enterprise products in his previous roles.

Ranjan brings extensive experience and expertise in all areas of engineering and product development, having built high-performing, highly-scalable, highly available and highly transnational enterprise products. Ranjan has more than 9 years of experience in product development and holds a Bachelors of Computer Engineering from NIT, Kurukshetra