Difference between revisions of "Mapgen memory optimisations"

From Minetest Developer Wiki
Jump to navigation Jump to search
m (fix typo)
(Update for MT v5.0.0)
Line 10: Line 10:
 
Localise the noise object outside <code>register_on_generated()</code> and initialise it to <code>nil</code>.
 
Localise the noise object outside <code>register_on_generated()</code> and initialise it to <code>nil</code>.
 
See the code below for how to create it once only.
 
See the code below for how to create it once only.
The creation of the perlin noise tables with <code>get3dMap_flat()</code> etc. is done per mapchunk
+
The creation of the perlin noise tables with <code>get_3d_map_flat()</code> etc. is done per mapchunk
  
 
<source>
 
<source>
 
local nobj_terrain = nil
 
local nobj_terrain = nil
 
local nobj_biome = nil
 
local nobj_biome = nil
...
+
-- ...
...
+
-- ...
 
-- On generated function
 
-- On generated function
 
minetest.register_on_generated(function(minp, maxp, seed)
 
minetest.register_on_generated(function(minp, maxp, seed)
   ...
+
   -- ...
   ...
+
   -- ...
 
   nobj_terrain = nobj_terrain or minetest.get_perlin_map(np_terrain, chulens3d)
 
   nobj_terrain = nobj_terrain or minetest.get_perlin_map(np_terrain, chulens3d)
 
   nobj_biome = nobj_biome or minetest.get_perlin_map(np_biome, chulens2d)
 
   nobj_biome = nobj_biome or minetest.get_perlin_map(np_biome, chulens2d)
 
    
 
    
   nobj_terrain:get3dMap_flat(minpos3d, nvals_terrain)
+
   nobj_terrain:get_3d_map_flat(minpos3d, nvals_terrain)
   nobj_biome:get2dMap_flat(minpos2d, nvals_biome)
+
   nobj_biome:get_2d_ap_flat(minpos2d, nvals_biome)
   ...
+
   -- ...
   ...
+
   -- ...
 
end)
 
end)
 
</source>
 
</source>
Line 42: Line 42:
 
    
 
    
 
   #### Methods
 
   #### Methods
   * `get2dMap_flat(pos, buffer)`: returns a flat `<size.x * size.y>` element array of 2D noise
+
   * `get_2d_map_flat(pos, buffer)`: returns a flat `<size.x * size.y>` element array of 2D noise
 
   with values starting at `pos={x=,y=}`
 
   with values starting at `pos={x=,y=}`
   * `get3dMap_flat(pos, buffer)`: Same as `get2dMap_flat`, but 3D noise
+
   * `get_3d_map_flat(pos, buffer)`: Same as `get_2d_map_flat`, but 3D noise
  
 
Localise the noise buffer outside <code>register_on_generated()</code>
 
Localise the noise buffer outside <code>register_on_generated()</code>
Use the buffer parameter in <code>get3dMap_flat()</code> etc.
+
Use the buffer parameter in <code>get_3d_map_flat()</code> etc.
  
 
<source>
 
<source>
Line 53: Line 53:
 
local nvals_terrain = {}
 
local nvals_terrain = {}
 
local nvals_biome = {}
 
local nvals_biome = {}
...
+
-- ...
...
+
-- ...
 
-- On generated function
 
-- On generated function
 
minetest.register_on_generated(function(minp, maxp, seed)
 
minetest.register_on_generated(function(minp, maxp, seed)
   ...
+
   -- ...
   ...
+
   -- ...
   nobj_terrain:get3dMap_flat(minpos3d, nvals_terrain)
+
   nobj_terrain:get3d_map_flat(minpos3d, nvals_terrain)
   nobj_biome:get2dMap_flat(minpos2d, nvals_biome)
+
   nobj_biome:get2d_map_flat(minpos2d, nvals_biome)
   ...
+
   -- ...
   ...
+
   -- ...
 
end)
 
end)
 
</source>
 
</source>
Line 91: Line 91:
 
-- On generated function
 
-- On generated function
 
minetest.register_on_generated(function(minp, maxp, seed)
 
minetest.register_on_generated(function(minp, maxp, seed)
   ...
+
   -- ...
   ...
+
   -- ...
 
   local vm, emin, emax = minetest.get_mapgen_object("voxelmanip")
 
   local vm, emin, emax = minetest.get_mapgen_object("voxelmanip")
 
   local area = VoxelArea:new{MinEdge = emin, MaxEdge = emax}
 
   local area = VoxelArea:new{MinEdge = emin, MaxEdge = emax}
 
   vm:get_data(data)
 
   vm:get_data(data)
   ...
+
   -- ...
   ...
+
   -- ...
 
end)
 
end)
 
</source>
 
</source>

Revision as of 10:11, 9 September 2019

Mbox information.png This page contains community-written advice, tips, tricks or recommendations about modding. Your mileage may vary.

Retranscription of Paramat's forum topic.

Perlin noise objects: Only create once

The noise object is created by minetest.get_perlin_map(). It has to be created inside register_on_generated() to be usable, but only needs to be created once, many mapgen mods create it for every mapchunk, this consumes memory unnecessarily.

Localise the noise object outside register_on_generated() and initialise it to nil. See the code below for how to create it once only. The creation of the perlin noise tables with get_3d_map_flat() etc. is done per mapchunk

local nobj_terrain = nil
local nobj_biome = nil
-- ...
-- ...
-- On generated function
minetest.register_on_generated(function(minp, maxp, seed)
   -- ...
   -- ...
   nobj_terrain = nobj_terrain or minetest.get_perlin_map(np_terrain, chulens3d)
   nobj_biome = nobj_biome or minetest.get_perlin_map(np_biome, chulens2d)
   
   nobj_terrain:get_3d_map_flat(minpos3d, nvals_terrain)
   nobj_biome:get_2d_ap_flat(minpos2d, nvals_biome)
   -- ...
   -- ...
end)

Perlin noise tables: Re-use a single table

The Lua table that stores the noise values for a mapchunk is big, especially for 3D noise (80 ^ 3 = 512000 values). Many Lua mapgens are creating a new table for every mapchunk, while the previous tables are only cleared out slowly by garbage collection, resulting in a large and unnecessary memory use.

A buffer parameter was added in 0.4.13 to avoid this, a single table is re-used by overwriting the former values.

  For each of the functions with an optional `buffer` parameter:  If `buffer` is not
  nil, this table will be used to store the result instead of creating a new table.
  
  #### Methods
  * `get_2d_map_flat(pos, buffer)`: returns a flat `<size.x * size.y>` element array of 2D noise
  with values starting at `pos={x=,y=}`
  * `get_3d_map_flat(pos, buffer)`: Same as `get_2d_map_flat`, but 3D noise

Localise the noise buffer outside register_on_generated() Use the buffer parameter in get_3d_map_flat() etc.

-- Localise noise buffers
local nvals_terrain = {}
local nvals_biome = {}
-- ...
-- ...
-- On generated function
minetest.register_on_generated(function(minp, maxp, seed)
   -- ...
   -- ...
   nobj_terrain:get3d_map_flat(minpos3d, nvals_terrain)
   nobj_biome:get2d_map_flat(minpos2d, nvals_biome)
   -- ...
   -- ...
end)

Lua voxelmanip table: Re-use a single table

The Lua table that stores the node content ids for a mapchunk plus the mapblock shell is big (112 ^ 3 = 1404928 values). Many Lua mapgens are creating a new table for every mapchunk, while the previous tables are only cleared out slowly by garbage collection, resulting in a large and unnecessary memory use.

A 'buffer' parameter was added in 0.4.13 to avoid this, a single table is re-used by overwriting the former values.

Very recently a 'buffer' parameter was also added to 'get_param2_data()', so is only usable there in latest Minetest dev or in Minetest 0.4.15 stable or later.

  * `get_data([buffer])`: Retrieves the node content data loaded into the `VoxelManip` object
      * returns raw node data in the form of an array of node content IDs
      * if the param `buffer` is present, this table will be used to store the result instead
  * `get_param2_data([buffer])`: Gets the raw `param2` data read into the `VoxelManip` object
      * Returns an array (indices 1 to volume) of integers ranging from `0` to `255`
      * If the param `buffer` is present, this table will be used to store the result instead

Localise the data buffer outside register_on_generated(). Use the buffer in get_data() or get_param2_data().

-- Localise data buffer
local data = {}

-- On generated function
minetest.register_on_generated(function(minp, maxp, seed)
   -- ...
   -- ...
   local vm, emin, emax = minetest.get_mapgen_object("voxelmanip")
   local area = VoxelArea:new{MinEdge = emin, MaxEdge = emax}
   vm:get_data(data)
   -- ...
   -- ...
end)