Software Integration, Batch Processing, Connectors: An Encyclopedia of Concepts in Automation

February 8, 2023

From software integration, to connectors, to batch processing, this encyclopedia introduces you to foundational concepts in enterprise automation.

From software integration, to batch processing, to connectors, this encyclopedia introduces you to some foundational concepts in automation.

Related: What you need to know about business process integration (BPI)

Types of Integration and Automation

Software integration lays the foundation for automation.

Software Integration

Software that’s been designed separately needs to be integrated for information to move between applications. Businesses today use many SaaS apps, databases, and microservices; it’s important to integrate applications in order to keep accounts in sync across applications and to automate processes across multiple applications. Software integration lays the foundation for automation. 

Enterprise application integration facilitates enterprise automation.

Enterprise Application Integration

Enterprise Application Integration refers to the process or infrastructure of integrating the multiple applications across an enterprise that power business processes. Enterprise application integration has changed dramatically in the cloud era. Although some traditional tools still use on-premise deployment, Workato’s platform, which performs both enterprise application integration and business process automation, use cloud-based API connectors.

Enterprise automation should be part of your digital transformation strategy.

Enterprise Automation

Enterprise Automation automates processes across the whole ecosystem of SaaS apps, microservices, data, connected devices, bots, and intelligent technologies that power an enterprise. Enterprise Automation combines the power of enterprise application integration, with wall-to-wall automation capabilities that support use-cases ranging from ETL and ELT, to bidirectional data syncs between SaaS apps, to workflow automations across a range of applications and microservices. Enterprise automation breaks down the dichotomy of integration v. automation by offering both in one platform. It can connect to AI and ML tools, as well as RPA, via API connections. 

3 clouds connected to each other by dotted lines.

iPaaS

iPaaS, or integration Platform as a Service, is an integration platform that organizations can use on a subscription basis. There is significant variety between the different iPaaS tools on the market. Traditional iPaaS requires technical expertise, and may require an on-premise deployment and server provisioning (referred to as ‘atoms’ or ‘nodes’). It’s traditionally deployed and maintained by the IT department. Enterprise automation tools differ from traditional iPaaS tools by not only offering robust integration capabilities, but also powering wall-to-wall automation of business processes.

How Individual Recipes Work

Recipes are Workato's term for automations.

Recipe

A recipe is an automation designed in the Workato platform. Recipes are assembled by dragging and dropping actions into a sequence. The actions usually correspond to one or more API calls. Users add business logic that determines how data is manipulated or transformed between applications. Users also add trigger actions that set off sequences of events in the automation.

Connectors make software integration and enterprise automation possible.

Connector

A connector is a pre-coded API connector that can be used in the process of building an automated workflow. 

Connector SDK

The connector SDK is a software development kit that developers can use to make custom connectors.

Business logic determines how a recipe works.

Business Logic

Business logic are the criteria that determine how business processes are conducted in an enterprise. For example, IF [X] event occurs, and [Y] conditions are present, [Z] event should follow. Business logic may include boolean operators like and, or, not, or and not. For example, IF [X] event occurs, and [A] conditions but not [B] conditions are present, [Z] event should occur. Business logic is used to determine the automated workflow that comprises business process automations. Business logic makes it possible for an automated process to mimic the human decision-making process.  

Open source recipes provide a base of knowledge that moves the community forward together. They can be used for software integration, business process automation, and more.

Open Source Automation Library

An open source automation library is a pool of recipes that have been contributed by users in the community. These community recipes can be copied and adapted for new use-cases. Many businesses have very similar business processes, such as PTO approval, lead routing, sending out hire letters with Docusign, updating the CRM after a sales order is fulfilled, or syncing account information Salesforce and Netsuite. The automation library concept makes it possible for organizations to build off existing knowledge and deploy automations easier and more quickly than they would by building automations from square one. 

Recipe cloning is a unique invention that makes the automation library work.

Recipe Cloning

Recipe cloning is the process of copying a recipe from the recipe library to be used elsewhere. When a recipe is cloned, the versioning process differentiates between logic changes and schema changes, so a user can make schema changes without altering the recipe logic. When a new user connects their app, the platform automatically introspects their app schema. That process identifies whether the fields and / or objects used in the recipe exist, and makes it easy to adjust the automation to work with your app schema. This is unique from traditional “automation templates”, which are rigid and have undifferentiated logic and data schema. The differentiation of the data and logic schema is a unique concept that facilitates recipe-sharing in the integration community

What Type of Systems and Processes Might Be Involved?

SaaS apps are important to businesses today, and are part of a larger shift toward cloud rather than on-prem computing.

SaaS Apps

SaaS stands for Software as a Service. SaaS apps are cloud-based software that companies can subscribe to without buying the software. 

On-prem, or on-premise deployment is not cloud-based. It requires server provisioning.

On-prem

On-premise, or on-prem, refers to a software, database, or integration platform that is not cloud-based. On-prem integration is less scalable than cloud-based integration, because it requires server provisioning, longer deployment times, and requires more time and expertise to modify.

Batch processing involves moving a lot of data at once.

Batch Processing

Batch processing is a more traditional process for syncing or processing data. With batch processing, you wait until a certain amount of data has been collected, or a certain amount of time has passed to perform the operation. For example, a company might use a nightly or weekly batch sync to move transaction records from an application to their data warehouse. With batch processing, companies update their data periodically rather than in real-time.

Real-time triggers fire a recipe as soon as the event is picked up.

Real-Time

There are four types of triggers: Polling, Real time, Scheduled or Bulk. Both polling and real time triggers are instigated by trigger events, but real time triggers fire a recipe as soon as the event is picked up. Real-time operations offer many benefits to businesses. For example, real-time operations ensure that decisions are being made from current rather than outdated data, because the single source of truth for data across the enterprise is updated in real-time. Additionally, in order to implement workflow automations that work across applications and departments at the pace at which business is conducted, organizations need their automated processes to respond in real-time to trigger events. 

APIs, or application programming interfaces, are used to connect and automate processes across cloud and on-prem applications.

APIs

An API is an application programming interface. Many powerful business tools, including AI tools like IBM’s Watson or Google Cloud Vision, can be used in automated workflows by connecting to their APIs. API connectors can be used to securely read, sync, transmit and transform data between applications. 

RPA is one type of automation tool. It is UI-based.

Related: The top 7 benefits from using API integration

RPA

RPA, or robotic process automation, is a UI-based automation technology. RPA uses screen-scraping to complete processes that would otherwise be done by a human worker, like manual data entry. Because RPA is UI-based, it is brittle to changes to its conditions. Although RPA cannot handle a wide range of automation use-cases for an enterprise (in fact, Forbes reports that  “A study by consulting firm Deloitte last year found that over 90% of RPA deployments fail to scale successfully,”), RPA performs some specific tasks well. For example, RPA can be used to extract information from a legacy application on a mainframe, where the UI conditions will remain consistent. Enterprise automation tools can connect to RPA via API to incorporate RPA into automated workflows.

Broader Concepts in Automation

Artificial intelligence is a power tool that businesses today can take advantage of.

Artificial Intelligence for Business Systems

Artificial intelligence can be used in business systems by connecting API-based artificial intelligence tools like Google Cloud Vision and IBM’s Watson into automated workflows. Artificial intelligence can currently be used to perform tasks like parsing voicemails to text or parsing images to text. When introduced as part of an automated workflow, these tools can be very powerful. In fact, organizations can use voicemail parsing to replace an entire call center, or use image parsing to entirely replace manual data entry tasks such as entering competitive wage research information from newspaper listings into their HR database. 

Enterprise automation facilitates scaling processes.

Scaling with Automation

Automation is useful for scaling business processes efficiently. Automation scales processes by reducing labor and manual data entry, accelerating processes, reducing human error and manual data entry, and performing actions in real-time based on trigger events. Scaling processes with automation facilitates organizations achieving personalization at scale, reducing data silos, maximizing the value of business data, and processing a high volume of transactions. 

Middleware has shaped the trajectory of integration and automation technology.

Middleware

Middleware acts as “plumbing” between different applications. Middleware makes it possible for information to move between applications. Now, the term generally refers to older forms of software integration. However, the concepts behind middleware have informed the trajectory toward the development of the modern enterprise automation tools that are on the market today. 

The 4th Industrial Revolution is bringing about transformational change to the way life and business happen.

4th Industrial Revolution

Artificial intelligence, machine learning, software integration, the Internet of Things, advancing automation technology, and ubiquitous connectivity define the contemporary economic and industrial climate. Salesforce describes the 4th Industrial Revolution as “the blurring of boundaries between the physical, digital, and biological worlds. It’s a fusion of advances in artificial intelligence (AI), robotics, the Internet of Things (IoT), 3D printing, genetic engineering, quantum computing, and other technologies.” 

Workato is an enterprise automation platform that combines robust, cloud-based enterprise application integration with enterprise automation. To learn more, request a demo from our team.