![\"\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/database_size.jpg\")
<\/a><\/p>\n<\/a>System Performance?<\/h2>\nArchiving reduces the size of the database and\u00a0prevents the growth. The goal is to lower the database costs and improve response times. SAP recommends to include the archiving already in the initial implementation plan. Decisions this stage can affect later archiving. For example, if you want to save the export invoices for a longer period than the others, it makes sense to\u00a0create own billing type for these.<\/p>\n
<\/a>Version upgrade, conversions?<\/h2>\nAt some point a version upgrade becomes actual. SAP recommends archiving before upgrading in order to reduce the volume of transaction data. This results to shorter runtimes and production downtime. I have once participated in an upgrade that had to be interrupted. Because of the size of the database a weekend was too short.<\/p>\n
<\/a>Considering HANA?<\/h2>\nIt pays off to archive before a HANA Project. Although HANA is rapid and can handle tons of data, it does not mean that you should move old, useless data there. First of all, HANA is expensive. Memory is much more expensive than disc space.\u00a0\u00a0SAP charges for HANA based on volume. A smaller database incurs lower hardware costs and helps speed up the migration process. Less documents to convert also enables shorter timelines for the project, from testing to the live conversion.<\/p>\n
<\/a>Legal Compliance \u2013 GDPR?<\/h2>\nEU General Data Protection Regulation strengthens the data protection for all individuals within the European Union. Personal data must be deleted after the primary purpose of the processing has ended unless it has to kept because of other legislation – such as tax legislation. Access to it must be blocked or restricted, and it must be kept only for the duration of the longest legal retention period, after which it must be deleted as stated in GDPR.<\/p>\n
![\"ILM_GDPR\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/ILM_GDPR-1024x325.jpg\")
<\/a><\/p>\nTo help with this task,\u00a0as of SAP NetWeaver 7.40, SAP Business Suite applications provide blocking and deletion functionality that is based on SAP Information Lifecycle Management (SAP ILM)<\/strong>.<\/p>\nData archiving is the cornerstone of SAP ILM.<\/p>\n
![\"Archiving_project_phases\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/Archiving_project_phases.jpg\")
<\/a><\/p>\n<\/a>Phases of Archiving Project<\/h1>\nSAP archiving involves complicated technical and business issues. To facilitate the communication between IT and business a project organization is best choice for the implementation. Project organization enables a better understanding and faster results.<\/p>\n
<\/a>Preparation<\/h2>\nArchiving is not solely an IT project. It affects the business processes and user\u2019s work. There are a lot of laws and regulations that govern the corporate information. These vary in different countries. The compliance to these is the responsibility of the business.<\/p>\n
Although it is a common project, it pays off to prepare the project inside IT before kicking off and involving the business. They are not interested in the technical details, they are only interested in what will happen to their data.<\/p>\n
<\/a>Initial analysis (IT)<\/h3>\nSAP archiving is probably novelty also for IT, why it is a good idea to engage an archiving consultant as a couch.<\/p>\n
Identify and analyze:<\/p>\n
\n- biggest tables and their growth rate<\/li>\n
- archiving objects for these tables<\/li>\n
- methods for data avoidance<\/li>\n
- dependencies between archiving objects in business processes<\/li>\n
- technical and business requirements<\/li>\n
- laws and regulations<\/li>\n
- access and reporting requirements<\/li>\n<\/ul>\n
<\/a>Identifying the biggest tables and their growth rate<\/h3>\nAs the main goal is to reduce the size if the database, the first thing to do is to find out what is in there. Which are the biggest tables and fastest growing tables? For this you have tools like SAP EWA reports, Data Volume Management (DVM) in Solution Manager or transaction\u00a0DB02 or DB02OLD.<\/p>\n
Which the biggest tables are depends on the business activities. This example below is from a consumer products company with heavy use of sales and logistics.<\/p>\n
![\"DB02OLD_biggest_tables\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/DB02OLD_biggest_tables.jpg\")
<\/a><\/p>\nQuick Wins<\/strong><\/p>\nAmong the biggest tables, you will find huge technical tables containing spool data, application and change logs etc. In the above example table TST03 contains spool data. These are easy targets for archiving or deleting, because they do not contain data important to business. Also\u00a0watch that\u00a0your service provider regularly empties the spools and deletes old background jobs.<\/p>\n
![\"QuickWin_tables\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/QuickWin_tables.jpg\")
<\/a><\/p>\n<\/a>Data avoidance<\/h3>\nInstead of collecting data, it is also possible to avoid it. Before deciding to archive you should analyze, whether there are any possibilities for this.<\/p>\n
![\"Avoid_data\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/Avoid_data.jpg\")
<\/a><\/p>\n\u00a0<\/strong><\/p>\n\n- Data prevention <\/strong>\u2014 Stop the creation of unnecessary data<\/li>\n<\/ol>\n
It pays off to\u00a0explore, whether tables you don\u2019t need, gather data. SAP Archiving guides (Data Management Guide for SAP Business Suite) and notes help to identify them. It is also possible to deactivate some technical parameters, which\u00a0generate unwanted and unnecessary volume.<\/p>\n
\n- Data aggregation <\/strong>\u2014 Reduce unnecessary details by summarizing certain data<\/li>\n<\/ol>\n
SAP offers different summarization options. Aggregates reduce the data volume. For example, FI document summary allows you to summarize\u00a0the material from the data coming from MM and SD. Without the summarization the accounting document could easily have hundreds of lines.<\/p>\n
![\"data_cleaning\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/data_cleaning.jpg\")
<\/a><\/p>\n\n- Data deletion <\/strong>\u2014 Eliminate any data that is not necessary for business or legal purposes<\/li>\n<\/ol>\n
There are\u00a0transactions for deleting (reorganizing) administrative and technical data from SAP. If you want to delete application data or master data, you soon find out that it cannot be done.\u00a0 It is done with archiving. But does it really make sense to store useless data like purchase requisitions that never matured to purchase orders?<\/p>\n
When you implement ILM, you can\u00a0also use archiving to clean old and unnecessary data. There is a new option Data Destruction, which deletes the archiving files.<\/p>\n
![\"ILM_actions\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/ILM_actions-300x97.jpg\")
<\/a><\/p>\n<\/a>Where to store the archived data?<\/h3>\nThe archived data is saved in a compressed form. The files are saved in the SAP file system, which is not a safe place to store\u00a0important archives. The archive files are moved from SAP to external long-term archive. For this purpose, an archiving system needs to be procured. There are several vendors and alternatives for SAP certified archive systems.<\/p>\n
Before project start you need to have the decision about the storage. It is important that the entire solution is at place, when you start the archiving tests. External archive systems are not cheap, which can be a problem.<\/p>\n
![\"Archiving](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/ArchivingRun.jpg\")
<\/a><\/p>\n<\/a>Implementation partner<\/h3>\nMost likely you want engage an implementation partner in the project. It is important to choose a partner that really\u00a0knows archiving. It can be difficult to find a partner with comprehensive knowledge. Often the offerings concentrate just to the technical aspects of archiving and the knowledge of business requirements can be narrow.<\/p>\n
\u00a0<\/b><\/p>\n<\/a>Project Team<\/h2>\nWhen you are clear with the preparations and have a vision of your project, it is time to prepare business for the coming and kick off the project.<\/p>\n
Keep in mind that it is a business project, not an IT Project.\u00a0Business owns the data.<\/p>\n
IT is responsible for technical configuration of archiving objects, archiving info structures, archiving order and other technical areas. The main contribution of business is to define the retention periods both in SAP and the archive system. This information is vital for application specific configuration of archiving objects.<\/p>\n
<\/a>Data Analysis<\/h2>\nIn this phase you take a closer look at the data. Focus on largest ERP tables. Depending on sap version and the business, the largest tables vary, but very often you find these tables among the ten biggest.<\/p>\n
\n- GLPCA \u2013 profit center line items<\/li>\n
- FAGLFLEXA – new general ledger<\/li>\n
- BSIS \u2013 open items<\/li>\n
- BKPF, BSEG \u2013 accounting document<\/li>\n
- MKPF, MSEG \u2013 material document<\/li>\n
- COEP \u2013 controlling documents<\/li>\n
- VBAK, VBRK, LIKP \u2013 SD order, deliveries, billing documents<\/li>\n<\/ul>\n
There usually one archiving object for each table. The\u00a0archiving object\u00a0specifies precisely which data is archived and how. Transaction DB15 enables you to determine the archiving objects. Below the archiving object for SD Billing documents.<\/p>\n
![\"Archiving_Object_SD_VBRK\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/Archiving_Object_SD_VBRK-300x239.jpg\")
<\/a><\/p>\n<\/a>Dependencies<\/h3>\nSAP archiving has lots of controls that ensure that data is fit for archiving is archived in right time and right order. For example, it is not possible to archive accounting documents that contain open items. A production order cannot be removed before the status is \u2018Closed\u2019. A sales order cannot be archived before all related deliveries documents are archived. Archiving validates that the business process is complete. If it is not, archiving fails. The reason\u00a0can be faulty configuration or missing user action.<\/p>\n
Take for example billing. It is not an isolated business transaction as you can see in the SD document flow.<\/p>\n
![\"Sales_process_archiving_objects\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/Sales_process_archiving_objects.jpg\")
<\/a><\/p>\nThere is an archiving object for each phase in the document flow.<\/p>\n
![\"SD_Document_flow\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/SD_Document_flow.jpg\")
<\/a>![\"SD_VBAK_archiving_sequence\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/SD_VBAK_archiving_sequence.jpg\")
<\/a><\/p>\nThere is also a defined sequence that must be followed when archiving these documents. Luckily you don\u2019t have to know it, the archiving objects will check that their predecessors are archived before them.<\/p>\n
Archiving master data (material, customers, and vendors) can be very painful. Prerequisite for archiving is that all transactions related to them has been archived. A material in an industrial company can have huge amount of dependencies from purchasing, production, \u00a0maintenance and sales.<\/p>\n
<\/a>Conceptual Design<\/h2>\nOne of the project results should be an archiving handbook<\/strong>, containing instructions and documentation of the process with all details. Start the document before testing and complement it during it. The archiving hand books should not remain a dead document on the shelf.<\/p>\nYou need to create a test plan and implementation plan.<\/p>\n
<\/a>Continuous archiving<\/h3>\nArchiving should be part of normal system maintenance and not firefighting. You need to plan:<\/p>\n
\n- who takes care of the monthly, weekly, yearly archiving runs and who takes care of the archives?<\/li>\n
- who checks the archiving logs to get the best result of the archiving? Who detects the erroneous ways of working that lead to incomplete processes and unarchivable data? Who informs business about faulty postings?<\/li>\n
- who takes care\u00a0of new archiving needs? For example, implementation of new modules and functionalities or organization and process changes that influence archiving?<\/li>\n<\/ul>\n
<\/a>Testing<\/h2>\nArchiving requires thorough testing. End-users should be key resources in the test phase. Testing is an essential\u00a0part of data archiving project.<\/p>\n
At this stage you should also be able to show the users how to access the archived data. For some archiving objects the archived data is displayed with same transactions as unarchived. This is the case in accounting and purchase documents. Some applications have own transactions for archived data (e.g. Display archived invoice).\u00a0 All participants should\u00a0understand and accept the consequences\u00a0of archiving in production\u00a0before the\u00a0decision is made.<\/p>\n
Archiving must be planned and done correctly from the beginning. The knowledge grows with experience. You cannot test and validate too much. It is too late to regret, when the data is archived. How to explain the auditor that he cannot get the invoices, because you screwed up the archiving.<\/p>\n
![\"traffic_lights\"](\"https:\/\/www.esfoor.fi\/en\/wp-content\/uploads\/sites\/3\/2018\/02\/traffic_lights-300x111.jpg\")
<\/a><\/p>\nThe initial tests often reveal that archiving is not going to be as simple as expected. In the SAP implementation all kinds of mistakes have been made. Accounts had been defined for \u201dopen items accounts\u201d but were never cleared.\u00a0 Sales document flows are not completed. There are open returns and open deliveries.\u00a0 This data cannot not be archived. The sales document flows can be so complicated that after tackling one error, tens of new errors appear.<\/p>\n
The cause of these errors is that the users did something wrong and must be informed and trainred to avoid them in the future. Afterwards it is very difficult to improve the quality of the data and there is no motivation to correct old documents. This is often also impossible.<\/p>\n
SAP has some corrections programs that can be used to force the documents to\u201d complete\u201d. However, these are not easy to find.<\/p>\n
\u00a0<\/b><\/p>\n
Don\u2019t get greedy and try to handle too many archiving objects at a time. Divide the project in smaller parts.<\/p>\n
Archiving runs can be very heavy and system intensive. To get a picture of the archiving times you need also to test the performance. However, remember that test and production systems do not have same load. Usually you should schedule the archiving runs to times with low system activity (nights, weekends).<\/p>\n
Archiving does not automatically improve the response times. The freed space becomes available after reorganization of the archived tables and indexes. It is IT\u2019s responsibility to take care of this.<\/p>\n
<\/a>The results of the project<\/h2>\nThe\u00a0main result of a successful archiving project is the first archiving in the production. It is often common that the first archiving is clean-up archiving,\u00a0which removes\u00a0a lot of old data.<\/p>\n
If you leave the archiving to this, after some time you find yourself in the same situation starting a new archiving project, where most of the work must be redone. For this reason, one of the project result should be a plan for continuous archiving.<\/p>\n
<\/a>Experiences<\/h1>\nTheory is often different from practice. Technically archiving is pretty simple, but the process is not. There are lots of obstacles on the way.<\/p>\n
Continuous archiving is a big challenge. When you search for resources, a vacuum is quickly formed around you. Also, many service providers are reluctant to offer archiving service. The unfortunate outcome is that someone from the IT department does the archiving on their own time. Some archiving runs can be automated using process chains, but also this requires supervision.<\/p>\n
It can be very difficult to sell the project to business. Not only does it waste the valuable time of users, but also needs expensive equipment. When the work starts, there are usually a lot of \u2018no-shows\u2019 that have more important things to do.<\/p>\n
Explain the consequences for not archiving. What it means\u00a0both for performance and maintenance costs.<\/p>\n
At some point a version upgrade becomes actual. SAP recommends archiving before upgrading in order to reduce the volume of transaction data. This results to shorter runtimes and production downtime. I have once participated in an upgrade that had to be interrupted. Because of the size of the database a weekend was too short.<\/p>\n
<\/a>Considering HANA?<\/h2>\nIt pays off to archive before a HANA Project. Although HANA is rapid and can handle tons of data, it does not mean that you should move old, useless data there. First of all, HANA is expensive. Memory is much more expensive than disc space.\u00a0\u00a0SAP charges for HANA based on volume. A smaller database incurs lower hardware costs and helps speed up the migration process. Less documents to convert also enables shorter timelines for the project, from testing to the live conversion.<\/p>\n
<\/a>Legal Compliance \u2013 GDPR?<\/h2>\nEU General Data Protection Regulation strengthens the data protection for all individuals within the European Union. Personal data must be deleted after the primary purpose of the processing has ended unless it has to kept because of other legislation – such as tax legislation. Access to it must be blocked or restricted, and it must be kept only for the duration of the longest legal retention period, after which it must be deleted as stated in GDPR.<\/p>\n
<\/a><\/p>\nTo help with this task,\u00a0as of SAP NetWeaver 7.40, SAP Business Suite applications provide blocking and deletion functionality that is based on SAP Information Lifecycle Management (SAP ILM)<\/strong>.<\/p>\nData archiving is the cornerstone of SAP ILM.<\/p>\n
<\/a><\/p>\n<\/a>Phases of Archiving Project<\/h1>\nSAP archiving involves complicated technical and business issues. To facilitate the communication between IT and business a project organization is best choice for the implementation. Project organization enables a better understanding and faster results.<\/p>\n
<\/a>Preparation<\/h2>\nArchiving is not solely an IT project. It affects the business processes and user\u2019s work. There are a lot of laws and regulations that govern the corporate information. These vary in different countries. The compliance to these is the responsibility of the business.<\/p>\n
Although it is a common project, it pays off to prepare the project inside IT before kicking off and involving the business. They are not interested in the technical details, they are only interested in what will happen to their data.<\/p>\n
<\/a>Initial analysis (IT)<\/h3>\nSAP archiving is probably novelty also for IT, why it is a good idea to engage an archiving consultant as a couch.<\/p>\n
Identify and analyze:<\/p>\n
\n- biggest tables and their growth rate<\/li>\n
- archiving objects for these tables<\/li>\n
- methods for data avoidance<\/li>\n
- dependencies between archiving objects in business processes<\/li>\n
- technical and business requirements<\/li>\n
- laws and regulations<\/li>\n
- access and reporting requirements<\/li>\n<\/ul>\n
<\/a>Identifying the biggest tables and their growth rate<\/h3>\nAs the main goal is to reduce the size if the database, the first thing to do is to find out what is in there. Which are the biggest tables and fastest growing tables? For this you have tools like SAP EWA reports, Data Volume Management (DVM) in Solution Manager or transaction\u00a0DB02 or DB02OLD.<\/p>\n
Which the biggest tables are depends on the business activities. This example below is from a consumer products company with heavy use of sales and logistics.<\/p>\n
<\/a><\/p>\nQuick Wins<\/strong><\/p>\nAmong the biggest tables, you will find huge technical tables containing spool data, application and change logs etc. In the above example table TST03 contains spool data. These are easy targets for archiving or deleting, because they do not contain data important to business. Also\u00a0watch that\u00a0your service provider regularly empties the spools and deletes old background jobs.<\/p>\n
<\/a><\/p>\n<\/a>Data avoidance<\/h3>\nInstead of collecting data, it is also possible to avoid it. Before deciding to archive you should analyze, whether there are any possibilities for this.<\/p>\n
<\/a><\/p>\n\u00a0<\/strong><\/p>\n\n- Data prevention <\/strong>\u2014 Stop the creation of unnecessary data<\/li>\n<\/ol>\n
It pays off to\u00a0explore, whether tables you don\u2019t need, gather data. SAP Archiving guides (Data Management Guide for SAP Business Suite) and notes help to identify them. It is also possible to deactivate some technical parameters, which\u00a0generate unwanted and unnecessary volume.<\/p>\n
\n- Data aggregation <\/strong>\u2014 Reduce unnecessary details by summarizing certain data<\/li>\n<\/ol>\n
SAP offers different summarization options. Aggregates reduce the data volume. For example, FI document summary allows you to summarize\u00a0the material from the data coming from MM and SD. Without the summarization the accounting document could easily have hundreds of lines.<\/p>\n
<\/a><\/p>\n\n- Data deletion <\/strong>\u2014 Eliminate any data that is not necessary for business or legal purposes<\/li>\n<\/ol>\n
There are\u00a0transactions for deleting (reorganizing) administrative and technical data from SAP. If you want to delete application data or master data, you soon find out that it cannot be done.\u00a0 It is done with archiving. But does it really make sense to store useless data like purchase requisitions that never matured to purchase orders?<\/p>\n
When you implement ILM, you can\u00a0also use archiving to clean old and unnecessary data. There is a new option Data Destruction, which deletes the archiving files.<\/p>\n
<\/a><\/p>\n<\/a>Where to store the archived data?<\/h3>\nThe archived data is saved in a compressed form. The files are saved in the SAP file system, which is not a safe place to store\u00a0important archives. The archive files are moved from SAP to external long-term archive. For this purpose, an archiving system needs to be procured. There are several vendors and alternatives for SAP certified archive systems.<\/p>\n
Before project start you need to have the decision about the storage. It is important that the entire solution is at place, when you start the archiving tests. External archive systems are not cheap, which can be a problem.<\/p>\n
<\/a><\/p>\n<\/a>Implementation partner<\/h3>\nMost likely you want engage an implementation partner in the project. It is important to choose a partner that really\u00a0knows archiving. It can be difficult to find a partner with comprehensive knowledge. Often the offerings concentrate just to the technical aspects of archiving and the knowledge of business requirements can be narrow.<\/p>\n
\u00a0<\/b><\/p>\n<\/a>Project Team<\/h2>\nWhen you are clear with the preparations and have a vision of your project, it is time to prepare business for the coming and kick off the project.<\/p>\n
Keep in mind that it is a business project, not an IT Project.\u00a0Business owns the data.<\/p>\n
IT is responsible for technical configuration of archiving objects, archiving info structures, archiving order and other technical areas. The main contribution of business is to define the retention periods both in SAP and the archive system. This information is vital for application specific configuration of archiving objects.<\/p>\n
<\/a>Data Analysis<\/h2>\nIn this phase you take a closer look at the data. Focus on largest ERP tables. Depending on sap version and the business, the largest tables vary, but very often you find these tables among the ten biggest.<\/p>\n
\n- GLPCA \u2013 profit center line items<\/li>\n
- FAGLFLEXA – new general ledger<\/li>\n
- BSIS \u2013 open items<\/li>\n
- BKPF, BSEG \u2013 accounting document<\/li>\n
- MKPF, MSEG \u2013 material document<\/li>\n
- COEP \u2013 controlling documents<\/li>\n
- VBAK, VBRK, LIKP \u2013 SD order, deliveries, billing documents<\/li>\n<\/ul>\n
There usually one archiving object for each table. The\u00a0archiving object\u00a0specifies precisely which data is archived and how. Transaction DB15 enables you to determine the archiving objects. Below the archiving object for SD Billing documents.<\/p>\n
<\/a><\/p>\n<\/a>Dependencies<\/h3>\nSAP archiving has lots of controls that ensure that data is fit for archiving is archived in right time and right order. For example, it is not possible to archive accounting documents that contain open items. A production order cannot be removed before the status is \u2018Closed\u2019. A sales order cannot be archived before all related deliveries documents are archived. Archiving validates that the business process is complete. If it is not, archiving fails. The reason\u00a0can be faulty configuration or missing user action.<\/p>\n
Take for example billing. It is not an isolated business transaction as you can see in the SD document flow.<\/p>\n
<\/a><\/p>\nThere is an archiving object for each phase in the document flow.<\/p>\n
<\/a><\/a><\/p>\nThere is also a defined sequence that must be followed when archiving these documents. Luckily you don\u2019t have to know it, the archiving objects will check that their predecessors are archived before them.<\/p>\n
Archiving master data (material, customers, and vendors) can be very painful. Prerequisite for archiving is that all transactions related to them has been archived. A material in an industrial company can have huge amount of dependencies from purchasing, production, \u00a0maintenance and sales.<\/p>\n
<\/a>Conceptual Design<\/h2>\nOne of the project results should be an archiving handbook<\/strong>, containing instructions and documentation of the process with all details. Start the document before testing and complement it during it. The archiving hand books should not remain a dead document on the shelf.<\/p>\nYou need to create a test plan and implementation plan.<\/p>\n
<\/a>Continuous archiving<\/h3>\nArchiving should be part of normal system maintenance and not firefighting. You need to plan:<\/p>\n
\n- who takes care of the monthly, weekly, yearly archiving runs and who takes care of the archives?<\/li>\n
- who checks the archiving logs to get the best result of the archiving? Who detects the erroneous ways of working that lead to incomplete processes and unarchivable data? Who informs business about faulty postings?<\/li>\n
- who takes care\u00a0of new archiving needs? For example, implementation of new modules and functionalities or organization and process changes that influence archiving?<\/li>\n<\/ul>\n
<\/a>Testing<\/h2>\nArchiving requires thorough testing. End-users should be key resources in the test phase. Testing is an essential\u00a0part of data archiving project.<\/p>\n
At this stage you should also be able to show the users how to access the archived data. For some archiving objects the archived data is displayed with same transactions as unarchived. This is the case in accounting and purchase documents. Some applications have own transactions for archived data (e.g. Display archived invoice).\u00a0 All participants should\u00a0understand and accept the consequences\u00a0of archiving in production\u00a0before the\u00a0decision is made.<\/p>\n
Archiving must be planned and done correctly from the beginning. The knowledge grows with experience. You cannot test and validate too much. It is too late to regret, when the data is archived. How to explain the auditor that he cannot get the invoices, because you screwed up the archiving.<\/p>\n
<\/a><\/p>\nThe initial tests often reveal that archiving is not going to be as simple as expected. In the SAP implementation all kinds of mistakes have been made. Accounts had been defined for \u201dopen items accounts\u201d but were never cleared.\u00a0 Sales document flows are not completed. There are open returns and open deliveries.\u00a0 This data cannot not be archived. The sales document flows can be so complicated that after tackling one error, tens of new errors appear.<\/p>\n
The cause of these errors is that the users did something wrong and must be informed and trainred to avoid them in the future. Afterwards it is very difficult to improve the quality of the data and there is no motivation to correct old documents. This is often also impossible.<\/p>\n
SAP has some corrections programs that can be used to force the documents to\u201d complete\u201d. However, these are not easy to find.<\/p>\n
\u00a0<\/b><\/p>\n
Don\u2019t get greedy and try to handle too many archiving objects at a time. Divide the project in smaller parts.<\/p>\n
Archiving runs can be very heavy and system intensive. To get a picture of the archiving times you need also to test the performance. However, remember that test and production systems do not have same load. Usually you should schedule the archiving runs to times with low system activity (nights, weekends).<\/p>\n
Archiving does not automatically improve the response times. The freed space becomes available after reorganization of the archived tables and indexes. It is IT\u2019s responsibility to take care of this.<\/p>\n
<\/a>The results of the project<\/h2>\nThe\u00a0main result of a successful archiving project is the first archiving in the production. It is often common that the first archiving is clean-up archiving,\u00a0which removes\u00a0a lot of old data.<\/p>\n
If you leave the archiving to this, after some time you find yourself in the same situation starting a new archiving project, where most of the work must be redone. For this reason, one of the project result should be a plan for continuous archiving.<\/p>\n
<\/a>Experiences<\/h1>\nTheory is often different from practice. Technically archiving is pretty simple, but the process is not. There are lots of obstacles on the way.<\/p>\n
Continuous archiving is a big challenge. When you search for resources, a vacuum is quickly formed around you. Also, many service providers are reluctant to offer archiving service. The unfortunate outcome is that someone from the IT department does the archiving on their own time. Some archiving runs can be automated using process chains, but also this requires supervision.<\/p>\n
It can be very difficult to sell the project to business. Not only does it waste the valuable time of users, but also needs expensive equipment. When the work starts, there are usually a lot of \u2018no-shows\u2019 that have more important things to do.<\/p>\n
Explain the consequences for not archiving. What it means\u00a0both for performance and maintenance costs.<\/p>\n
EU General Data Protection Regulation strengthens the data protection for all individuals within the European Union. Personal data must be deleted after the primary purpose of the processing has ended unless it has to kept because of other legislation – such as tax legislation. Access to it must be blocked or restricted, and it must be kept only for the duration of the longest legal retention period, after which it must be deleted as stated in GDPR.<\/p>\n
To help with this task,\u00a0as of SAP NetWeaver 7.40, SAP Business Suite applications provide blocking and deletion functionality that is based on SAP Information Lifecycle Management (SAP ILM)<\/strong>.<\/p>\n Data archiving is the cornerstone of SAP ILM.<\/p>\n SAP archiving involves complicated technical and business issues. To facilitate the communication between IT and business a project organization is best choice for the implementation. Project organization enables a better understanding and faster results.<\/p>\n Archiving is not solely an IT project. It affects the business processes and user\u2019s work. There are a lot of laws and regulations that govern the corporate information. These vary in different countries. The compliance to these is the responsibility of the business.<\/p>\n Although it is a common project, it pays off to prepare the project inside IT before kicking off and involving the business. They are not interested in the technical details, they are only interested in what will happen to their data.<\/p>\n SAP archiving is probably novelty also for IT, why it is a good idea to engage an archiving consultant as a couch.<\/p>\n Identify and analyze:<\/p>\n As the main goal is to reduce the size if the database, the first thing to do is to find out what is in there. Which are the biggest tables and fastest growing tables? For this you have tools like SAP EWA reports, Data Volume Management (DVM) in Solution Manager or transaction\u00a0DB02 or DB02OLD.<\/p>\n Which the biggest tables are depends on the business activities. This example below is from a consumer products company with heavy use of sales and logistics.<\/p>\n Quick Wins<\/strong><\/p>\n Among the biggest tables, you will find huge technical tables containing spool data, application and change logs etc. In the above example table TST03 contains spool data. These are easy targets for archiving or deleting, because they do not contain data important to business. Also\u00a0watch that\u00a0your service provider regularly empties the spools and deletes old background jobs.<\/p>\n Instead of collecting data, it is also possible to avoid it. Before deciding to archive you should analyze, whether there are any possibilities for this.<\/p>\n \u00a0<\/strong><\/p>\n It pays off to\u00a0explore, whether tables you don\u2019t need, gather data. SAP Archiving guides (Data Management Guide for SAP Business Suite) and notes help to identify them. It is also possible to deactivate some technical parameters, which\u00a0generate unwanted and unnecessary volume.<\/p>\n SAP offers different summarization options. Aggregates reduce the data volume. For example, FI document summary allows you to summarize\u00a0the material from the data coming from MM and SD. Without the summarization the accounting document could easily have hundreds of lines.<\/p>\n There are\u00a0transactions for deleting (reorganizing) administrative and technical data from SAP. If you want to delete application data or master data, you soon find out that it cannot be done.\u00a0 It is done with archiving. But does it really make sense to store useless data like purchase requisitions that never matured to purchase orders?<\/p>\n When you implement ILM, you can\u00a0also use archiving to clean old and unnecessary data. There is a new option Data Destruction, which deletes the archiving files.<\/p>\n The archived data is saved in a compressed form. The files are saved in the SAP file system, which is not a safe place to store\u00a0important archives. The archive files are moved from SAP to external long-term archive. For this purpose, an archiving system needs to be procured. There are several vendors and alternatives for SAP certified archive systems.<\/p>\n Before project start you need to have the decision about the storage. It is important that the entire solution is at place, when you start the archiving tests. External archive systems are not cheap, which can be a problem.<\/p>\n Most likely you want engage an implementation partner in the project. It is important to choose a partner that really\u00a0knows archiving. It can be difficult to find a partner with comprehensive knowledge. Often the offerings concentrate just to the technical aspects of archiving and the knowledge of business requirements can be narrow.<\/p>\n \u00a0<\/b><\/p>\n When you are clear with the preparations and have a vision of your project, it is time to prepare business for the coming and kick off the project.<\/p>\n Keep in mind that it is a business project, not an IT Project.\u00a0Business owns the data.<\/p>\n IT is responsible for technical configuration of archiving objects, archiving info structures, archiving order and other technical areas. The main contribution of business is to define the retention periods both in SAP and the archive system. This information is vital for application specific configuration of archiving objects.<\/p>\n In this phase you take a closer look at the data. Focus on largest ERP tables. Depending on sap version and the business, the largest tables vary, but very often you find these tables among the ten biggest.<\/p>\n There usually one archiving object for each table. The\u00a0archiving object\u00a0specifies precisely which data is archived and how. Transaction DB15 enables you to determine the archiving objects. Below the archiving object for SD Billing documents.<\/p>\n SAP archiving has lots of controls that ensure that data is fit for archiving is archived in right time and right order. For example, it is not possible to archive accounting documents that contain open items. A production order cannot be removed before the status is \u2018Closed\u2019. A sales order cannot be archived before all related deliveries documents are archived. Archiving validates that the business process is complete. If it is not, archiving fails. The reason\u00a0can be faulty configuration or missing user action.<\/p>\n Take for example billing. It is not an isolated business transaction as you can see in the SD document flow.<\/p>\n There is an archiving object for each phase in the document flow.<\/p>\n There is also a defined sequence that must be followed when archiving these documents. Luckily you don\u2019t have to know it, the archiving objects will check that their predecessors are archived before them.<\/p>\n Archiving master data (material, customers, and vendors) can be very painful. Prerequisite for archiving is that all transactions related to them has been archived. A material in an industrial company can have huge amount of dependencies from purchasing, production, \u00a0maintenance and sales.<\/p>\n One of the project results should be an archiving handbook<\/strong>, containing instructions and documentation of the process with all details. Start the document before testing and complement it during it. The archiving hand books should not remain a dead document on the shelf.<\/p>\n You need to create a test plan and implementation plan.<\/p>\n Archiving should be part of normal system maintenance and not firefighting. You need to plan:<\/p>\n Archiving requires thorough testing. End-users should be key resources in the test phase. Testing is an essential\u00a0part of data archiving project.<\/p>\n At this stage you should also be able to show the users how to access the archived data. For some archiving objects the archived data is displayed with same transactions as unarchived. This is the case in accounting and purchase documents. Some applications have own transactions for archived data (e.g. Display archived invoice).\u00a0 All participants should\u00a0understand and accept the consequences\u00a0of archiving in production\u00a0before the\u00a0decision is made.<\/p>\n Archiving must be planned and done correctly from the beginning. The knowledge grows with experience. You cannot test and validate too much. It is too late to regret, when the data is archived. How to explain the auditor that he cannot get the invoices, because you screwed up the archiving.<\/p>\n The initial tests often reveal that archiving is not going to be as simple as expected. In the SAP implementation all kinds of mistakes have been made. Accounts had been defined for \u201dopen items accounts\u201d but were never cleared.\u00a0 Sales document flows are not completed. There are open returns and open deliveries.\u00a0 This data cannot not be archived. The sales document flows can be so complicated that after tackling one error, tens of new errors appear.<\/p>\n The cause of these errors is that the users did something wrong and must be informed and trainred to avoid them in the future. Afterwards it is very difficult to improve the quality of the data and there is no motivation to correct old documents. This is often also impossible.<\/p>\n SAP has some corrections programs that can be used to force the documents to\u201d complete\u201d. However, these are not easy to find.<\/p>\n \u00a0<\/b><\/p>\n Don\u2019t get greedy and try to handle too many archiving objects at a time. Divide the project in smaller parts.<\/p>\n Archiving runs can be very heavy and system intensive. To get a picture of the archiving times you need also to test the performance. However, remember that test and production systems do not have same load. Usually you should schedule the archiving runs to times with low system activity (nights, weekends).<\/p>\n Archiving does not automatically improve the response times. The freed space becomes available after reorganization of the archived tables and indexes. It is IT\u2019s responsibility to take care of this.<\/p>\n The\u00a0main result of a successful archiving project is the first archiving in the production. It is often common that the first archiving is clean-up archiving,\u00a0which removes\u00a0a lot of old data.<\/p>\n If you leave the archiving to this, after some time you find yourself in the same situation starting a new archiving project, where most of the work must be redone. For this reason, one of the project result should be a plan for continuous archiving.<\/p>\n Theory is often different from practice. Technically archiving is pretty simple, but the process is not. There are lots of obstacles on the way.<\/p>\n Continuous archiving is a big challenge. When you search for resources, a vacuum is quickly formed around you. Also, many service providers are reluctant to offer archiving service. The unfortunate outcome is that someone from the IT department does the archiving on their own time. Some archiving runs can be automated using process chains, but also this requires supervision.<\/p>\n It can be very difficult to sell the project to business. Not only does it waste the valuable time of users, but also needs expensive equipment. When the work starts, there are usually a lot of \u2018no-shows\u2019 that have more important things to do.<\/p>\n Explain the consequences for not archiving. What it means\u00a0both for performance and maintenance costs.<\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a>Phases of Archiving Project<\/h1>\n
<\/a>Preparation<\/h2>\n
<\/a>Initial analysis (IT)<\/h3>\n
\n
<\/a>Identifying the biggest tables and their growth rate<\/h3>\n
<\/a><\/p>\n<\/a><\/p>\n<\/a>Data avoidance<\/h3>\n
<\/a><\/p>\n\n
\n
<\/a><\/p>\n\n
<\/a><\/p>\n<\/a>Where to store the archived data?<\/h3>\n
<\/a><\/p>\n<\/a>Implementation partner<\/h3>\n
<\/a>Project Team<\/h2>\n
<\/a>Data Analysis<\/h2>\n
\n
<\/a><\/p>\n<\/a>Dependencies<\/h3>\n
<\/a><\/p>\n<\/a><\/a><\/p>\n<\/a>Conceptual Design<\/h2>\n
<\/a>Continuous archiving<\/h3>\n
\n
<\/a>Testing<\/h2>\n
<\/a><\/p>\n<\/a>The results of the project<\/h2>\n
<\/a>Experiences<\/h1>\n