I was recently reading about IBM InfoSphere Optim Test Data Management solutions. I was trying to understand the importance of a Test Data Management solution in general. In this blog I will share some of my gatherings on this topic.
Test Data Management (TDM) is about the provisioning of data for non-production environments, especially for test purposes but also for development, training, quality assurance, demonstrations or other activities. Test data has always been required to support application development and other environments. It can be achieved in an ad hoc brute force way by
- copying some production data
- cloning entire production databases
- creating some sample from the model
Copying data or cloning production databases has a number of drawbacks. To begin with: how do organisations manage costs? Copying production data or cloning production databases is largely a manual process. Moreover it requires a whole new set of hardware and software, along with the licenses to match, and must be duplicated across each testing environment. As a result you can easily end up with multiple copies of the same database across multiple projects in development.
Alternatively, you can have multiple development teams sharing the same test database but sharing means that there is often contention for resources, resulting in extended delivery schedules. Indeed, getting access to the right data at the right time can be a major issue regardless of whether you share test data or have multiple test databases. It takes time to generate (copy or clone) new datasets because databases administrators, who typically complete this task, have other priorities. Test teams often have to wait several days, or even longer, to get test data. In agile environments, in particular, manual copying or cloning can slow development. There are additional challenges with copying production data or cloning production databases if the application processes any sensitive data that developers or testers should not see.
Another approach to test data creation is for developers or testers that we mentioned above is to create their own. But it is difficult to ensure that the test data accurately represents the production data. Moreover it is a hassle to keep creating new test datasets as the data model may have changed.
TDM introduces an automated way of generating appropriate test data that can support agile environments and protect sensitive data. While there is obviously a licensing cost for TDM solutions, they increase testing agility, reduce risk, ensure thorough testing against an appropriately broad range of tests, decrease hardware and software costs, provide compliance through data protection, and reduce development cycles so that new applications can be brought into production faster and with fewer bugs.