SAP Housekeeping – How much of your SAP (HANA) data is valuable?
Introduction
Whether you are already on SAP HANA or planning to move to HANA this question is important. According to Forrester research “Only 12-18% of all data in BW is actually used“. SAP BW systems generate high amounts of temporary data that lose value quickly after creation, but SAP ERP systems have these problems too. Often 6 out of the 10 biggest SAP ERP tables are not creating significant business value. See below the output of an SAP Early-watch report. Only 4 out of 10 biggest tables in this SAP ERP system contain transactional data that is directly needed to run this business. The rest of the data is important too, but should be purged from the system on a regular basis after end of purpose.
Types of temporary SAP data and housekeeping
Temporary SAP data is all data created on daily basis, that loses its value quickly after its creation. Temporary data is often found in these areas:
• logs (application logs, change logs)
• staging (PSA & Changelogs)
• communication protocols (IDOCS, RFC logs),
• administration data (requests, change documents)
• and others.
The regular clean-up of temporary SAP data is called Housekeeping. The document Data Management Guide for SAP Business Suite has more than 200 pages!. This shows that this topic is quite significant.
An efficient data management strategy is an important part of trying to maintain good system performance and keep the total cost of your system in check. This guide covers the entire component spectrum of SAP Business Suite and SAP NetWeaver.
“My system does not fit into my SAP HANA box”. The impact of temporary data on complex SAP landscapes
If your users are complaining about system performance, your operational costs are high, and your SAP database is running out of space, you may be looking for guidance on best practice for SAP system housekeeping. Here are some symptoms when running SAP systems with significant amounts of aged SAP data.
1. SAP system is running out of database space
In SAP BW especially the temporary data can consume up to 20-40% of the system size. Regular deletion and clean-up of data after the end of its purpose can keep data growth under control.
2. SAP system response is too slow
Temporary data is stored in tables that are accessed in nearly all system operations. A typical example is an application log, which is accessed (write or read) by most operations in the SAP system. If tables containing application logs are not cleaned up regularly and grow significantly, this will have a negative impact on system performance.
3. Moving to HANA with too much baggage
Deletion of temporary data (housekeeping) is part of every HANA migration. It’s unlikely someone would be willing to migrate all SAP legacy data to HANA. Clean-up of aged logs and staging data is a must do activity to fit into an optimal HANA box environment. These activities are inevitable when migrating into a Suite on HANA, S4/HANA, BW on HANA or BW/4HANA.
Best practice on handling temporary SAP data in complex SAP landscapes
Step one is to find out how fit your system is, considering the amount of housekeeping data. This can be done via an SAP Early-watch report or by alternative services focused on SAP data management. Once the potential for housekeeping is uncovered the best practices for keeping the data growth under control are:
1. Housekeeping should be done REGULARLY
Handling temporary data is not a one-time activity. As data is generated on a daily basis the clean-up must be done with same or similar frequency.
2. Housekeeping should be done AUTOMATICALLY
Regular housekeeping is a repetitive, time consuming administration task. Automation is the key to keep the efforts in balance with the positive effects of housekeeping.
3. CENTRAL documentation, definition of rules, scheduling and monitoring of housekeeping
SAP landscapes are complex IT environments. Every system within the landscape contains temporary data and should be part of the housekeeping activity. Having central rules, one place for set up, execution, monitoring and documentation can ensure the balance between maintenance of housekeeping and positive effects of it.
4. Housekeeping activities should be UPDATED
With new releases and system upgrades such as S/4HANA and BW/4HANA, you need to also update corresponding housekeeping activities. The Data Management Guide for SAP Business Suite – is a must read for everybody who wants to get the right balance between the cost of SAP data and the value it generates.
5. Housekeeping should be done SAFELY
Housekeeping is a set of archiving and deletion activities. Setting up retention times should be done safely, initially with a conservative approach with finetuning cycles. Storing deleted temporary data with reduced operational costs is usually the safest way. This includes storing data in Hadoop clusters outside of SAP systems before this gets deleted. Storing SAP temporary data after end of purpose outside of SAP or in a compressed form, brings significant benefit – more aggressive thresholds keeping the security on the same level.
Expert tips: Typical quick wins
The price of every gigabyte of space within SAP HANA landscapes is motivation enough to run data management projects. With the HANA migration wave with S4/HANA, BW on HANA and BW4/HANA it is no longer topic that can be postponed. There are many SAP customers facing similar challenges and often solutions have repeating patterns.
Here are few tips on saving a significant amount of database space in SAP HANA or traditional SAP systems on legacy database systems.
SOFFCONT1 table
The SOFFCONT1 table grows quickly in SAP ERP landscapes and often gets into the top 10 largest tables in the system (TIP: check the list of largest tables in DB02 transaction). This table belongs to SOC3 and contains SAP Business Workplace/SAPoffice Documents attachments. The size of SOFFCONT1 depends significantly on how these types of documents / attachments are used in the system and how they are stored. If external storage is not used, the SAP system stores the office attachments in the SOFFCONT1 table. That’s why it grows so quickly. To stop it from getting bigger, SAP recommends using an external storage system (e.g. archive storage). SOFFCONT1 is one of many SAP tables that contain temporary data, that is data that loses value quickly after its creation. These tables often grow rapidly because data is not handled properly after serving its purpose. My recommendations is to move SOFFCONT1 data outside of the SAP application server and move it to Hadoop cluster. Hadoop next to SAP has many other use cases, such as archiving, IoT, Big Data, Business Intelligence, etc. Storing SOFFCONT1 in Hadoop cluster connected to SAP could be your first use case to start with.
PSA and ChangeLog in SAP BW
The staging area in SAP BW systems could grow to up to 20-40% of the total system size. There could be a significant number of PSA and ChangeLogs that are not regularly cleaned up by the process chain, with staging data kept in the system for years. This has a negative impact on loading times and overall cost of ownership. PSA and ChangeLogs should be cleaned after 7 days with the option to keep them in compressed form in a so-called “RecycleBin” before the data gets out of purpose (usually within 90 days).
Summary
To summarize, housekeeping SAP systems (cleanup of aged SAP temporary data) is an important task that often doesn’t get the attention it deserves. It is important to clean up the data automatically, which not only reduces the overall effort, but also improves system performance and lowers storage costs. This is important whether you are on HANA or migrating to a Suite on HANA, S/4HANA, BW on HANA or BW4/HANA.
Author: Jan Meszaros – Service Line Manager at Datavard
Need to hire SAP Data Management resources? Looking for SAP Data Management work? Get in touch with Eursap – Europe’s Specialist SAP Recruitment Agency