Data Integration FAQs

The data integration process that DotActiv follows consists of eight steps and needs input from you, the customer.

The scoping meeting has multiple parts.

Firstly, DotActiv sends you all the necessary documentation. This includes information on the data integration process steps; a Data Feeds document and a Staging Tables document.

Data Feeds are a mechanism you would use to transport data from one location to another. An example of a data feed is a Microsoft SQL script.

Secondly, once you have read through all the documentation, you can send back any questions for DotActiv to answer.

Lastly, DotActiv sets up a meeting, usually done online via Google Hangouts to discuss any final needs.

You are expected to read through all the documentation provided.

Doing so allows you to educate yourself on the integration process that DotActiv follows. It also means you can ensure that you involve all the necessary people from the beginning.

From the client, DotActiv expects the chief business custodian of the DotActiv application to be present.

DotActiv also expects the IT custodian to be present too. That is not necessarily the person doing the work but rather the person who oversees everyone.

While not necessary, DotActiv recommends you include someone from your business intelligence division as well as someone from infrastructure.

You can expect a senior IT support technician, who is your main point of contact during integration.

DotActiv includes other authorised IT technicians if and when needed.

DotActiv prepares any pre-asked questions on the integration process and data feeds where necessary.

• 3Ghz dual-core CPU;
• 16GB Ram; and
• 300GB HDD.

If the server is under specced on processor or RAM and you have an extensive database with many users simultaneously hitting it, you can expect to experience performance issues.

If it is under specced on space, for example, the new data stops importing to your hard drive.

All the automated work also halts because of the hardware limitation.

DotActiv requests the connection details as well as company security policies.

If DotActiv needs a VPN account to connect to your business, for example, you must provide one along with installation instructions and configuration settings.

The connection details to the server include all the usernames and the passwords for the dedicated accounts for DotActiv to complete work.

Because the data is on your server, in your environment, and your network, it falls under your security policies.

DotActiv adheres to your security policies. If you give DotActiv a dedicated VPN with full-time access, DotActiv treats it with the utmost respect. Access is only available for authorised users.

If you decide that DotActiv can only get access on request, by VPN with someone supervising, DotActiv will comply.

DotActiv requires permanent remote access for ongoing support purposes. However, long-term, it's not necessary.

During the first few months, DotActiv actively monitors the integration to ensure it is stable and running smoothly.

Should there be any issues, DotActiv can immediately go onto your server and diagnose the problem.

You can restrict DotActiv's access to the server if you prefer to go this route. In that case, DotActiv complies with any security policies you have in place.

Please note, during the initial implementation, DotActiv requires permanent access because otherwise, the process to get everything up and running is difficult.

Because the server is in your environment and under your control, its deployment is your responsibility.

You are responsible for licensing it, managing it, and maintaining it.

That includes any non-DotActiv related applications such as the operating system, files, anti-virus, anti-malware, SQL, all the hardware, and the network.

DotActiv only maintains the content of the DotActiv databases as well as the DotActiv applications.

You need to contact your IT administrator as DotActiv does not carry out hardware support.

What DotActiv does offer is when there is a suspicion that the DotActiv software and hardware are not playing well together. In such instances, DotActiv assists as best as possible to work alongside your IT team to diagnose any problem.

The only person who has access to your data is the IT support consultant/s authorised to access your data for setup and support purposes.

The consultant is the one you meet during the scoping meeting.

Outside of that, if you have signed a services contract with DotActiv, it also includes the relevant space planners.

If the initial support consultant is busy when you need assistance, DotActiv assigns the work to the next available support consultant. While that consultant might not have been in the scoping meeting, they have the same authorisation by being part of the support consultant team.

Your data is as safe as the security policies you have in place. The data remains on your server, in your environment and network.

DotActiv won't copy anything off your server unless DotActiv has asked for a local backup for its developers to diagnose a problem.

Even then, DotActiv uses the data to develop a fix for a problem. After that, DotActiv discards the data.

It is always better for you to host the server because it's your data in your environment and under your scrutiny and security policies.

Under exceptional circumstances, DotActiv can assist with temporary hosting for up to six months. However, since it is not a core aspect of the business, DotActiv doesn't offer it.

The authorised IT support consultant agreed upon during the initial scoping meeting is responsible for deploying the DotActiv database.us

If you believe the deployment of the database is taking too long to complete, your point of contact is the IT support consultant who led the scoping meeting.

If they are busy or on leave, the next person to contact is the person with whom you signed the deal.

When you script your data into your staging tables, the DotActiv Import Service monitors the table for any new data. If there is new data, it processes that into the DotActiv database structure.

When done running, it flags it to show it has processed each record. It continuously monitors the data in two-minute cycles.

You can set it to run 24 hours a day or only for certain hours, be that during the business day or after hours.

It runs at the time you specify every two minutes to see if there is new data or not. If there is no new data, it then checks the next import.

Data imports include Markets, Products, Sales, Stock, and DC Stock.

It is usually one database but can be up to two.

The primary database is named 'DA' and your company name — for example, DARetailerName.

The 'DA' is in case the DotActiv database is on a shared server. The naming convention makes it easier for you to identify should there be other DotActiv databases on your server.

If you use StoreComs, you may also have this database on your server.

The staging tables needed are documented in the staging tables document, provided before the initial scoping meeting.

The minimum requirements are Markets, Products and Sales. While not necessary, DotActiv recommends you provide Stock data as well.

DotActiv itself doesn't need to deploy a database. You could, under instruction, deploy it yourself.

For DotActiv Pro and Enterprise, a database is a requirement. A database is required for integration.

The Planogram Import Utility is an application that extracts information from your DotActiv planograms.

The Import Utility extracts the data from your space plans such as product facings, shelves, pegboards, hanging bars, banners, and so on. This information is then imported to your DotActiv Database where you can report on it.

Your shelf planners will then have a specific folder on their machines where their FLPs will be saved to. The import utility will then push the FLP files to your store communications tool in Activ8, from where your stores can download and implement it.

The Data Import Utility works hand in hand with the DotActiv Import Service.

The Import Utility is the front-end application for configuring all imports. It configures which columns in the staging table must go into which value in the DotActiv software.

All these configurations happen in the Data Import Utility, and the Import Service handles the automation thereof.

You must provide DotActiv with administrative rights to complete the installation and set directory permissions.

DotActiv also requires a service account, set to those applications. The server account permanently runs these services.

If you don't want to provide DotActiv with a password because of security reasons, you can log in and assign that service account to those applications yourself.

You are responsible for populating the staging tables. Because you pull the data from your systems and push it into the table, it's also your responsibility to ensure the information is populated accurately and timeously.

When setting up and maintaining scripts, DotActiv does not get involved.

Should you have any questions as to the content of the data and the staging table as well as how it all works, DotActiv can assist.

Also, when it comes to populating the staging tables, DotActiv has the option where you can provide the CSV files in a specific location. DotActiv has scripts that extract it from the location to the staging table.

However, these scripts are pre-defined. DotActiv does not provide scripts to extract data from your systems.

DotActiv expects that you develop the scripts needed, populate the staging tables and complete the initial testing to ensure it meets the requirements stipulated in the Data Feeds document.

You must also not alter any of the staging tables as that affects the predefined settings. If you shift data or add it incorrectly, you are expected to correct it.

Also, any information you push into the staging tables is your responsibility. How this data translates into the DotActiv database is DotActiv's responsibility.

You must insert your data into the staging tables as per the predefined data types. This is all stipulated and explained in the Staging Tables document provided before the scoping meeting.

Should you be unable to insert your data directly into the staging tables, we can offer you a script which will process a CSV into the staging table. This CSV would require a predefined naming - TableName_Date.CSV, which would go to a predefined location.

DotActiv runs initial checks to ensure the data is in the correct format. However, it is not DotActiv's responsibility to ensure the data is correct.

If DotActiv finds an issue, it works with your IT team to resolve the issue. A solution includes clearing out your data so that you can re-script it.

If DotActiv finds that the error is because of your negligence, DotActiv can bill for this time at its discretion.

DotActiv doesn’t get involved in setting up or maintaining your scripts.

It is entirely dependent on you, your IT department and your in-house systems.

We explain Data Feeds here.

DotActiv conducts preliminary checks on your data. It includes the necessary sense checks to ensure it meets all requirements.

While these checks ensure all the data looks correct, it doesn’t guarantee the accuracy of the data within the staging tables.

The next testing takes place after the importation of data into the DotActiv database.

The testing includes looking at the data in the DotActiv database and ensuring it matches what is in your staging tables.

From there, DotActiv also performs basic application functionality. It's to ensure the application works as it should with the database.

DotActiv expects that when you consider the data within the DotActiv application, you agree with it. Are you happy with the data? Do you want to change or update a field?

From there, you must also check the data on your side to ensure its accuracy.

As the server is in your environment, it is your responsibility to ensure it is both stable and accurate.

You must ensure your data feeds are stable and accurate as DotActiv does not manage your scripts.

You must also be aware of your timings. For example, if you know you script your data on the fourth of every month, you should know that something is wrong if you don't see new data coming through on the fifth.

You should then contact your IT department to find out if the data did run as expected. If the data didn't run, you can contact DotActiv to investigate the issue.

If you are unsure of where the problem lies, DotActiv can assist. At the very least, DotActiv can point you in the right direction.

For example, DotActiv can check if there is a new data file or not. If not, you must contact your IT department.

Both you and DotActiv are responsible for signing off on the integration.

When DotActiv sends you the integration hand-over document, which includes all your mappings, validation reports, configuration settings for your user application, you must read through everything.

If you find any issues with it or have any questions, you must raise them.

It is your responsibility to communicate that you're happy and ready to sign off on the integration.

You must ensure all the mappings are correct. The validation report must be signed off, and the user of the application must be able to function and work correctly.

If you find any issues or are not happy with anything, you must contact DotActiv as soon as possible to correct any problems.

If any critical points are missing, DotActiv requests that the IT support consultant corrects the work. Should there be a need for escalation, DotActiv will ensure that this happens.

Should an error be found, you don’t have to start the integration process from the beginning.

Sometimes all it requires is a fresh batch of data. If that is the case, DotActiv redeploys everything so you can rerun your data.

Once you have signed off on the integration, that means you've gone live.

DotActiv expects you to treat the system as a live production system from this point forward. It is no longer a test site so you can't manipulate the data.

If you did attempt to manipulate the data, you could damage it. In that case, you would need to rescript and reprocess your data.

If you experience any issues at this stage, you must contact the IT support consultant assigned to your integration so they can resolve it.

Because of the nature of integration, DotActiv allows for a little bit of leeway up until official sign-off. Even after sign-off, there is leeway for the first six months. However, it is within reason.

After that, since you are full-fledged production environment, DotActiv is stricter.

There is no real expectation from DotActiv at this point.

It’s vital to communicate that you have signed off on the integration and are aware that the system is live.