Roll and Extended Memoryztta/roll_first - This is the first area of memory assigned to a work process. This parameter is usually set to 1 as a technically necessity as the modern applications will end up using more memory than the first roll segment. Even if ztta/roll_first is set to 1, the minimum amount required for administrative data is allocated from roll memory. It is best to leave the parameter at 1.
ztta/roll_extension - Further memory allocation to the work process is done from Extended Memory, after first roll segment is used up. ztta/roll_extension parameter determines how much memory can be used by a user context in the extended memory. In other words, this is the limit of the SAP Extended Memory for each work process. This is true in most cases, however, ztta/roll_extension is Extended Memory limit for a single "i-mode". Depending on the number of i-modes, normally up to 6, the value of maximum extended memory can raise up to 6*ztta/roll_extension theoretically. ztta/roll_extension must be smaller than (4096 - ztta/max_memreq_MB) * 1024 * 1024 - 1. ztta/roll_extension_dia and ztta/roll_extension_nondia are now available to tune the allocation separately to dialog work process and non-dialog work processes. If the work processes are using extended memory above this limit, see if the selection criteria can be reduced. If that is not feasible, then look to increase this parameter after confirming that the work process has indeed exceeded this limit and not em/initial_size_MB.
em/initial_size_MB - The extended memory is allocated when the SAP instance is started in accordance with the value of em/initial_size_MB. Each internal session can request extended memory up to a value of ztta/roll_extension in multiples of em/block_size_KB from the extended memory pool (em/initial_size_MB). When the SAP instance runs out of this memory, the work processes will start using heap memory causing them to go in to PRIV mode. Setting of em/initial_size_MB usually depends on the physical RAM on the server and if you have any other instances on that same server. Depending on available RAM this parameter can be set between 1024 MB and 1TB. em/initial_size_MB loses its significance if special memory handling eg ES/TABLE in AIX or es/implementation in Linux is activated.
abap/heap_area_total - When a dialog work process consumes Roll and Extended Memory available to it, it continues the transaction execution from heap memory. abap/heap_area_total restricts the total amount of heap memory that all WPs can use. If you set this parameter to 0, it will translate to 1 TB. This value should be decided based on swap space available to the OS.
abap/heap_area_dia or abap/heap_area_nondia - These parameters restrict the amount of heap memory (Private Memory) that an SAP dialog work process or non-dialog work process can allocate respectively. TSV_TNEW_* or SYSTEM_NO_MEMORY errors indicate that this limit is reached. If you set these parameters to 0, the kernel will translate it to 1 TB. On a 64 bit system these three parameters (abap/heap_area_*) can be safely set to high values such as 6GB.