By Daniel Martinez, Lead Software Engineer – Ibexlabs
By Yash Kimanti, Software Engineer – Ibexlabs
Every business has unique ways to approach the sales process. One commonality, however, is that customer data is an important driver to make a sales process successful.
To help businesses get the most out of their customer data, customer relationship management (CRM) tools are used widely to manage data throughout the customer journey.
In the sales process, organizations need to update the CRM with customer data to keep track of opportunities. It’s a challenging task, and some businesses keep a dedicated team to maintain it. The challenges become even more cumbersome when there’s a need to deal with multiple CRMs simultaneously.
In that case, the exchange of data between two CRMs should be seamless and glitch-free. For this, you need a level of technical proficiency to regularly update customer data on your CRM. These opportunities, in the form of fluctuating data, need to have a single source of truth between two systems. It’s a challenge to keep the data in sync all of the time.
For AWS Partners, the ACE Pipeline Manager is an exclusive benefit for participating members in the APN Customer Engagements (ACE) program. It provides organizations with full-service management of their pipeline of AWS customers, allowing for joint collaboration with Amazon Web Services (AWS) on those engagements.
In this post, we will discuss how various AWS services support Ibexlabs’ software-as-a-service (SaaS) solution, OppSync SPIM (Sales Pipeline Integration and Management), to better optimize and automate CRM integration with the ACE Pipeline Manager.
Ibexlabs is an AWS Competency Partner and Managed Service Provider (MSP) that provides next-generation solutions that equip customers for success on the AWS platform. Ibexlabs is also a security consultant partner helping many SMBs and startups in their cloud journey by focusing on security and compliance.
Challenges in Synchronizing Two CRM Systems
It can be challenging for businesses to ensure continuous data syncing between two CRM systems. When there’s an update on one of the CRMs and it doesn’t show on the other despite an integration, you need manual efforts to make this update.
One of the challenges during integration is assuring data is syncing to both CRMs, thus eliminating the need for manual intervention.
Other challenges may include:
- Lack of scalability: While scaling, downtime may cause a delay in syncing up. Additionally, scalability involves several steps from the user’s side such as coding, software updates, and monitoring.
- Improper administration: Maintaining two databases is a tough experience and may require coding to access a database at a technical level.
- Downtime: Duplicate and mixed up data may cause inconsistencies that require additional time and effort for cleanup, resulting in downtime.
- Security: Lack of industry-standard encryption may cause a threat to a user’s important data.
- Syncing inconsistencies: It’s a challenge for users to maintain the data lifecycle and keep the integration updated from end-to-end.
- Unable to share data pipeline: If a business is not able to share the data pipeline seamlessly, it may cause loss on shared opportunities.
CRM + ACE Integration with OppSync SPIM
A sales pipeline offers opportunity to visualize sales operations by focusing on potential buyers in progressive stages of their purchasing process.
With a sales pipeline, the sales team can measure how effective company sales efforts are when turning leads into customers. This data-driven process may require a company to integrate two CRM systems.
Designed using AWS services, OppSync SPIM (Sales Pipeline Integration and Management) from Ibexlabs integrates a partner’s existing CRM with the ACE Pipeline Manager without partners having to invest development time.
Users get instant notifications whenever there’s an update in either CRM systems. SPIM doesn’t save any data and takes security measures supported by AWS services to ensure privacy for the users.
AWS Partners can download a detailed report anytime. Additionally, it’s a no-code solution that means partners don’t have to put in effort towards its maintenance.
AWS Services Empowering OppSync SPIM
While designing the SPIM, Ibexlabs solved many challenges with the help of AWS services. In the diagram below, you can see the core architecture of OppSync and how AWS services such as Amazon ECS, AWS Lambda, and Amazon RDS were used to build the platform to help execute the sales pipeline integration.
AWS security services such as AWS KMS, WAF, and AWS Secrets Manager helped to ensure a strong security foundation for protection of customer information.
Figure 1 – Overview of AWS services behind OppSync SPIM.
Amazon Elastic Container Service (Amazon ECS) is a container orchestration platform that allows you to easily deploy any containerized workload in a serverless environment. You don’t need to maintain a control plane or node infrastructure common in other container orchestrators.
Early on, OppSync deployed most critical workloads on Docker through Amazon ECS, as the compute model provided by AWS Lambda didn’t fit a use case where Ibexlabs needed to continuously communicate with the many third-party systems OppSync had integrations for.
AWS Fargate, when used with Amazon ECS, is a fully managed compute engine that can be scaled up and downsized as needed. This elasticity is especially relevant to OppSync’s core engine as it needs to process and transmit variable amounts of data periodically to third-party systems through the integration it supports.
Another important feature of ECS that OppSync leverages is the ability to deploy as many instances of a given service as needed. This allows OppSync to distribute the requests between the replicas and ensure no single service gets overwhelmed with requests and becomes a single point of failure.
Features like high availability, elasticity, and concurrency led Ibexlabs to use Amazon ECS to deploy core parts of the OppSync solution, and drove the team to leverage Lambda for other critical system needs.
AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers and runtimes. It continuously scales, maintains a consistent performance at any scale, and optimizes for cost-efficiency.
OppSync uses Lambda to deploy several data processing pipelines that need to be run concurrently to transform and transmit massive amounts of data ingested from various sources into other target systems.
To enable the synchronization capabilities between ACE and users’ CRMs, OppSync uses Lambda to process data generated in Amazon Simple Storage Service (Amazon S3) for many different users. This data can vary in size for different users and is a great use case for parallel processing.
Another important aspect of this workflow is loading the data extracted and transformed from ACE into the different CRMs. By using Lambda, OppSync can easily parallelize this workload and have each invocation process different datasets for different AWS Partners at the same time without disrupting the service.
The team at Ibexlabs can iterate faster because Lambda supports sharing code between different functions by packaging it into a .zip file and uploading as a Lambda layer.
This lets OppSync share a common framework and toolset between functions, thus allowing developers at Ibexlabs to focus on writing actual business logic of product and not reinvent the instrumentation and abstractions they have already defined and tested through time.
Amazon SNS and Amazon SQS
Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS) are fully managed messaging services that implement a pub/sub communication and queueing system, respectively. Ibexlabs uses both services extensively to integrate various services and Lambda functions together.
By combining SNS and SQS, Ibexlabs can compose different services to create event-driven architectures that scale seamlessly. Since both services are fully managed, they are able to allocate resources as needed.
The key SNS feature leveraged by OppSync is the ability to spread messages to a large number of consumers to process messages in parallel. This allows the tool to batch and transform the data between supported integrations with efficiency.
To ensure resilience and apply backpressure when processing data is sent through SNS, OppSync leverages SQS to buffer messages. Because of this, a given service does not get overwhelmed by bursts of requests as it processes one message at a time.
To add another layer of redundancy, OppSync leverages SQS dead-letter queues to reprocess requests that failed in reaching their destination.
Overall, SNS and SQS are fundamental pieces that let Ibexlabs build scalable and resilient architectures while driving down costs to a fraction of the usual price when compared to managing such infrastructure ourselves.
Amazon Relational Database Service (Amazon RDS) is a managed service that enables OppSync to provision, manage, and scale a relational database.
Ibexlabs chose Amazon RDS early on during planning due to its ease of use and the ability to create high-performance replicas they could leverage to offload reads easily.
A key aspect of RDS that works in conjunction with replication is the ability to scale up and down resources with minimal downtime. This makes for an ideal disaster recovery setup, as OppSync can quickly promote a replica to be the main database and scale its resources in a matter of minutes to maintain our services.
Lastly, the ability to isolate databases within a virtual private cloud (VPC) and encrypt data at rest and in transit makes it ideal for this use case, as customer data privacy is a top priority for Ibexlabs, including the metadata stored on their databases.
AWS Key Management Service
Security, privacy, and regulatory compliance are top priorities as Ibexlabs needs to process users’ confidential business data. To address these challenges, the team leverages AWS Key Management Service (KMS), which is a fully managed service that makes it easy to create, manage, and use cryptographic keys across multiple AWS services.
Data encryption both at rest and in transit is important because users trust Ibexlabs with sensitive business information. The team leverages KMS to generate encryption keys for all of the services where data are stored.
Ibexlabs can audit the usage of those keys by leveraging AWS CloudTrail’s integration with KMS and meet regulatory requirements.
Web Application Firewall
AWS WAF is a managed firewall solution that helps secure applications against common attack patterns such as SQL injection or cross-site scripting. It allows you to create your own rules to block and filter traffic, and also provides a monitoring dashboard where you can have real-time visibility into your application’s web traffic.
Ibexlabs leverages AWS Managed Rules for AWS WAF extensively to guard OppSync against OWASP Top 10 security risks and other less-known ones. Since it provides robust monitoring and altering capabilities, Ibexlabs has set up alerts for any unwanted traffic pattern to help detect such traffic patterns and to enact an immediate mitigation plan.
Overall, the team at Ibexlabs likes the ease of use by which they can add or delete rules to block requests from certain regions or certain patterns detected, such as bots performing automated spamming.
Introducing a suitable technology empowers businesses to connect with potential buyers and convert them into opportunities.
In the sales pipeline, a buyer goes through various stages in the purchasing process and every stage has corresponding data. CRM systems manage this customer data, but sharing data between CRMs can be routinely difficult.
Ibexlabs used various AWS services to develop the OppSync SPIM platform and enabled the integration of users’ CRMs with the ACE Pipeline Manager with near real-time data syncing capabilities. This process helps AWS Partners by eliminating duplicate efforts when managing two sales pipelines.
The content and opinions in this blog are those of the third-party author and AWS is not responsible for the content or accuracy of this post.
Ibexlabs – AWS Partner Spotlight
Ibexlabs is an AWS Competency Partner and MSP that provides next-generation solutions that equip customers for success on the AWS platform.
*Already worked with Ibexlabs? Rate the Partner
*To review an AWS Partner, you must be a customer that has worked with them directly on a project.