Is your ETL service ready for Cloud?

CloudIn my previous blog, I mentioned how we can make an ETL job ready for cloud using Information Server. So the next questions would be
1. What are somethings to be considered from the perspective of deployment of such a service on Cloud?
2. Whether it is already being implemented or is this just a theory?

So in this blog, I will address some aspects of the first question.

After creation of a Data Transformation Service that can be launched on the Cloud, some of the following questions need to be considered. These are not exhaustive, but may be good starting point based on my experience on Cloud…

System and Topology Constraints

One of the primary reason to go on cloud is it provides Elasticity.

  • This means that we have to have an assumption of the number of VMs (Virtual Machines) are required initially.
  • How to monitor the Load in the VMs ? High load means the VM runs a danger of being pulled down.
  • How to Load balance the requests at VMs?
  • How to provision more VMs dynamically in case the Load increases?

User Experience:

  • How to keep track of the usage of the user and generate usage report for billing?
  • If the service crashes in between the operation how to give the High Availability experience to the user?
  • Once the service is up and running how to apply a patch without impacting the users?
  • What are the logs provided to the user in case the operation fails?

Security Risks

  • A developer can either accidentally or maliciously create service instances that will quickly allocate all of the available Service VMs and exhaust the capacity.
  • Is there anyway the data of the user be compromised maliciously? Note on Cloud the users are sharing these services.
  • Malicious software installed on VMs and engaging in unscrupulous activity, data destruction, information gathering or
    creation of backdoors.
  • Malicious vulnerability discovery in applications, services or protocol implementations.

One thought on “Is your ETL service ready for Cloud?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s