Difference between revisions of "TODO"

From Minetest Developer Wiki
Jump to navigation Jump to search
 
(145 intermediate revisions by 24 users not shown)
Line 1: Line 1:
−
=== Lua MapVoxelManipulator Interface ===
+
{{Mbox|type=issue|text='''Checkout the [https://github.com/minetest/minetest/blob/master/doc/direction.md roadmap] instead'''<br/>The roadmap outlines a consensus on medium-term Minetest development. This page contains other wishlist items from core developers, but hasn't necessarily reached a consensus and may be long term.}}
−
===== Description =====
+
 
−
Practically any mod that is more complex than the little clicky thing on the end of ink pens would want to modify portions of the map. To do this, a minetest.env:set_node() call is required for each node to be modified. It does not take much imagination to arrive at the conclusion that the performance of these mods suffer accordingly, due to not only the inherent slowness of Lua, but the overhead of Lua-to-C calls, and the overhead of each Map::setNode() call. To make matters even worse, all this must be done within an envlock, leaving the rest of the server unresponsive while the Lua mods do their thing. Absolutely disgusting.<br />
+
== TODO in Minetest ==
−
The answer is to give the Lua API the power of the ManualMapVoxelManipulator, so nodes may be retrieved or modified through a 3d array, and then blit (if necessary) back to the map.
+
=== Things to do soon ===
−
===== What needs to be done =====
+
 
−
* The API for this needs to be well thought-out - no afterthoughts or hacky looking pieces.
+
==== Mapgen ====
−
* The central structure of the VoxelManipulator will be the data itself - a 3D array (in Lua, a table containing tables containing tables of MapNodes), or even better, just one huge array, leaving the modders responsible for finding the index within the array (the latter would surely improve performance).
+
 
−
* An API to simply read chunks of map to an array without any intentions of modifying it.  An envlock would only be required when initially reading from the map to the VoxelManipulator.
+
* Add configuration for DungeonGen
−
* An API to read chunks of map, then blit back to the map when done.  An envlock is required throughout the lifetime of the object (unless you're not concerned with the contents of the map being modified in between the mod doing so).
+
* Add configuration for CaveGen
−
* An API to create a blank VoxelManipulator, perhaps filled with CONTENT_IGNORE, to which the mod sets only some blocks, and is blitted back to the map (CONTENT_IGNORE nodes aren't set, obviously). Here, an envlock would only be required when blitting back changes.
+
 
 +
==== DecorationDef ====
 +
 
 +
* Add L-system decoration support
 +
* Add cutoff handling
 +
 
 +
==== BiomeDefManager ====
 +
 
 +
* Add biomemap cache
 +
* Scale temperature and humidity noise to more sensible value ranges
 +
* Add biome weight attribute
 +
* Find out how to account for elevation better!
 +
* Add Voronoi diagram maker? https://forum.minetest.net/viewtopic.php?f=9&t=13163
 +
 
 +
==== MapVoxelManipulator ====
 +
 
 +
* Figure out a permanent, elegant solution to the walled-in cave problem - can use is_ground_content now - does this belong in VoxelManipulator?
 +
 
 +
==== EmergeThread ====
 +
 
 +
* Figure out the thread priority mess
 +
* Assign thread affinities for a bit of a performance boost (reduce L2 cache misses)
 +
 
 +
==== GUIs / Formspecs ====
 +
 
 +
* New main menu, which promotes Minetest better [https://github.com/minetest/minetest/issues/6733 6733]
 +
* Refactor formspec / GUI code to be independent from serialisation [https://github.com/minetest/minetest/issues/9358 9358]
 +
* Add new GUI API, probably based on a Lua DSL [https://github.com/minetest/minetest/issues/6527 6527]
 +
* New GUI default theme [https://github.com/minetest/minetest/issues/9802 9802]
 +
 
 +
==== Other ====
 +
 
 +
* Make a library for reading a Minetest map without worrying about the database used. (libmtmap?)
 +
* Add block tinting (grass, water, sky, etc.) (for biomes)
 +
* Add Fullbright option
 +
 
 +
=== Big, long term goals (Client) ===
 +
 
 +
==== Client-Side Scripting (CSM) ====
 +
See original plans: [[Client_scripting_plans]]
 +
 
 +
* Server-Sent Client-Side Modding (SSCSM)
 +
* Client-side prediction APIs
 +
* GUI APIs
 +
 
 +
==== Client-Side Biomes ====
 +
The server should transfer biome definitions to the client which calculates biomes locally.
 +
Crossing over into a new biome should trigger a client-side script callback
 +
Biomes have effects:
 +
* Full scene post-effects (tint everything a different color)
 +
* Temporarily override settings (cloud_height, enable_clouds, water_wave_height, water_wave_length, water_wave_speed)
 +
 
 +
==== Hardware Lighting for Light Sources ====
 +
 
 +
''' Description '''<br />
 +
Benefits:
 +
 
 +
* would allow variable light range and cutoff
 +
* would free up param1 of MapNodes to be used for storing more immediate node metadata (always a good thing!).
 +
* would allow colored light sources
 +
 
 +
Of course, can use dynamic lights, except they're very intensive and generally a scarce resource
 +
Dynamically generated lightmaps seem like a good way forward. <br />
 +
http://joshbeam.com/articles/dynamic_lightmaps_in_opengl/
 
<br />
 
<br />
−
=== LuaJIT ===
+
RealBadAngel is currently working on shader-based lighting. ''What strategy is he using?''
−
===== Description =====
 
−
Lua is an interpreted scripting language.  As such, it is *slow*.  No, really. Much of the time, it feels like the problem of Lua's slowness is the elephant in the room being avoided by adding more and more API so the actual work is delegated to the core in C++.  While some may argue that Lua mods are simply the dashboard to the car, and the core is well... the engine, it cannot be argued that the amount of work being deferred onto the core isn't much more than necessary.
 
−
===== What needs to be done =====
 
−
* Research LuaJIT.  Find out how it fits in with the current Lua interpreter, what changes would need to be made, etc.  Probably not too much.
 
−
* Real, substantial benchmarks.  Some within the Minetest community have commented that LuaJIT has been tried before, but the performance increases "weren't worth it", and was therefore dumped - this may not be so.  Regardless, even if the performance increase is minimal 'in most cases', it must be added to not neglect the subset of cases where the benefits would be much greater.  Also, 'less than optimal' is unacceptable, especially if the work to make it optimal had already been done.
 
 
<br />
 
<br />
−
=== Voxel Area Entities ===
 
−
===== Description =====
 
−
How cool would it be to build a full-sized boat in Minetest, and then drive it around the ocean?  That's one example of something that would be possible with this feature.<br />
 
−
Voxel Area Entities are moveable ActiveObjects whose mesh and texture is user-modifiable via setting MapNodes in it.  This is an extremely powerful feature, and while perhaps not of extreme difficulty to implement, there is very much to be done.
 
  
−
===== What needs to be done =====
+
==== Occlusion Culling/Occlusion Query ====
 +
 
 +
Irrlicht does support occlusion culling.  If enabled, it culls scene nodes which are entirely out of the camera's view fulcrum. <br />
 +
Problem:  The entire map is an Irrlicht scene node. ''Why? What was the rationale for this decision?''<br />
 +
Solution:  Make scene nodes per MapBlock. <br />
 +
While we're at it, playing around with scene nodes, it'd probably be a good idea to maintain two (or three) separate nodes for each MapBlock, one for each type of material.  Opaque, transparent, and liquid drawtypes.  This fixes the still unsolved material transparency problem.
 +
 
 +
=== Big, long term goals (Server) ===
 +
 
 +
==== Voxel Area Entities ====
 +
 
 +
''' Description '''<br />
 +
How cool would it be to build a full-sized boat in Minetest, and then drive it around the ocean? That's one example of something that would be possible with this feature.<br />
 +
Voxel Area Entities are moveable ActiveObjects whose mesh and texture is user-modifiable via setting MapNodes in it. This is an extremely powerful feature, and while perhaps not of extreme difficulty to implement, there is very much to be done.<br />
 +
''' What needs to be done '''
 
* A new derived SAO and CAO (they would need to be treated fundamentally differently), along with setNode() and removeNode() calls.
 
* A new derived SAO and CAO (they would need to be treated fundamentally differently), along with setNode() and removeNode() calls.
 
* A derived VoxelManipulator class, perhaps called ObjectVoxelManipulator, which will be used to manipulate them.
 
* A derived VoxelManipulator class, perhaps called ObjectVoxelManipulator, which will be used to manipulate them.
 
* These types of objects have meshes, and will need to be updated in MeshUpdateThread along with MapBlocks.
 
* These types of objects have meshes, and will need to be updated in MeshUpdateThread along with MapBlocks.
−
* Some way to store these in the map. Perhaps these objects will get special non-block-ID keys in the map DB.
+
* Some way to store these in the map. Perhaps these objects will get special non-block-ID keys in the map DB.
−
* A decent way to serialize these objects. This detail in particular will require special attention.
+
* A decent way to serialize these objects. This detail in particular will require special attention.
 +
* For the previous two points, see http://irc.minetest.net/minetest-dev/2013-10-21#i_3383926 for a discussion of a possible way to handle the objects.
 
<br />
 
<br />
−
=== Envlock Scope Reduction ===
+
 
−
===== Description =====
+
==== Envlock Scope Reduction ====
−
Envlock needs to be put on a diet. Seriously. This is believed to be the primary reason for perceived unresponsiveness in Minetest - solving this would help matters immensely.
+
 
−
===== What needs to be done =====
+
''' Description '''<br />
 +
Envlock needs to be put on a diet. Seriously. This is believed to be the primary reason for perceived unresponsiveness in Minetest - solving this would help matters immensely.<br />
 +
''' What needs to be done '''
 
* Add a thin abstraction layer for atomic operation intrinsics.
 
* Add a thin abstraction layer for atomic operation intrinsics.
 
* Replace Environment operations that require concurrency but are simple enough with the said atomic operations and clever ordering.
 
* Replace Environment operations that require concurrency but are simple enough with the said atomic operations and clever ordering.
 
* Add a thread-safe hashtable structure to be used instead of std::map (or core::map) where locks would be needed, make use of this.
 
* Add a thread-safe hashtable structure to be used instead of std::map (or core::map) where locks would be needed, make use of this.
−
* RCU for Map things - big structures such as MapBlocks could benefit from Read-Copy-Update for synchronization. This is the biggest change, but the reward would be immense.
+
* RCU for Map things - big structures such as MapBlocks could benefit from Read-Copy-Update for synchronization. This is the biggest change, but the reward would be immense.
 +
<br />
 +
 
 +
==== Pre-generate world ====
 +
 
 +
''' Description '''<br />
 +
''' What needs to be done '''
 
<br />
 
<br />
−
=== Bars ===
+
 
−
===== Description =====
+
==== Add colorlike to node param types ====
−
A set of Lua API to maintain graphical bars representing various values on the client's interface.  This would be the *right* way to more generally implement health, armor, breath, hunger, and XP.
+
 
−
===== What needs to be done =====
+
''' Description '''<br />
−
* Lua API to define the bars, remove, and set them.
+
Add a field like facedir, wallmounted, and liquidlike called colorlike that will allow the user to register a set of colors so that the color of some base node is modified on draw.  This would save many node definitions.<br />
−
* A general way to draw them (a la 'draw hearts' code in game.cpp, draw_hotbar())
+
This field would be (part of, at least) param2 in a MapNode.  If colorlike is specified on its own, then it can define and use up to 256 colors.  If used with facedir or wallmounted, then, it can only define and use 8 colors.
−
* Some sane boundaries so bars cannot be arbitrarily drawn all over the screen and overwrite other UI elements.
+
<br />
−
New network packets, to be sent from the server to the client:
+
''' What needs to be done '''<br />
−
* Add bar (bar identifier, name of texture for bar elements, title, initial value, et cetera.)
+
* Store a color LUT with the ContentFeatures for the defined node that is to be passed along from Lua when registering the node.
−
* Remove bar (bar identifier)
+
* Figure out how to draw these colors in inventory screens and what not without having to prerender more textures
−
* Set bar value (bar identifier, new value). Happens on a Lua defined-event (someone gets hit and takes damage, or w/e).
+
* Bump ContentFeatures version
 +
<br />
 +
 
 +
==== Rewrite falling code to C++ ====
 +
 
 +
''' Description '''<br />
 +
nodeupdate() now very slow and can't handle large amount of falling nodes.
 +
100-1000 falling nodes now makes server very slow, 10000+ cause segfault.
 +
To test - make sand floating island via mapgen, touch it
 +
<br />
 +
''' What needs to be done '''<br />
 +
Rewrite in core with queue (like Map::transformLiquids)
 +
<br />
 +
 
 +
==== Pathfinder ====
 +
See [[Pathfinder wishlist]].
 +
 
 +
=== Big protocol changes ===
 +
 
 +
There are several features that would probably bring benefits in the long term, but can't currently be merged because of network protocol incompatibility.
 +
That is, they either can't be implemented in a way that lets old clients connect to new servers and new clients to old servers, or it would be a major pain to do so.
 +
The point of this section is to list such features so that they can be merged at once: a one-time incompatibility, perhaps coinciding with the release of 0.5.0,
 +
is better than breaking compatibility all the time.
 +
'''Note: ''' Not all of the things listed below need to be done, these are just ideas.
 +
* A binary key-value storage which easily supports missing fields with default values (e.g. [https://github.com/celeron55/minetest/blob/meta_set_nodedef_2/src/util/template_serialize.h header], [https://github.com/celeron55/minetest/blob/meta_set_nodedef_2/src/util/template_serialize.cpp source], [https://github.com/celeron55/minetest/blob/meta_set_nodedef_2/src/nodedef.cpp#L327 usage])
 +
* Using TCP for reliable data transfer (e.g. [https://github.com/celeron55/minetest/tree/tcp_connection tcp_connection])
 +
* Make the client request mapblocks (e.g. [https://github.com/celeron55/minetest/tree/tcp_blocks_2 tcp_blocks_2])
 +
* Improved authentication and encryption
 +
* Remove old compatibility kludges.
 +
* Many command IDs have become obsolete in the protocol: they could be consolidated
 +
* Change builtin CONTENT_* to 0, 1, 2
 +
<br />
 +
 
 +
== See also ==
 +
* [https://wiki.minetest.net/Terminology Terminology]
 +
 
 +
[[Category:Core Engine]]
 +
[[Category:Proposal‏‎]]

Latest revision as of 13:45, 25 October 2022

Mbox important.png Checkout the roadmap instead
The roadmap outlines a consensus on medium-term Minetest development. This page contains other wishlist items from core developers, but hasn't necessarily reached a consensus and may be long term.

TODO in Minetest

Things to do soon

Mapgen

  • Add configuration for DungeonGen
  • Add configuration for CaveGen

DecorationDef

  • Add L-system decoration support
  • Add cutoff handling

BiomeDefManager

MapVoxelManipulator

  • Figure out a permanent, elegant solution to the walled-in cave problem - can use is_ground_content now - does this belong in VoxelManipulator?

EmergeThread

  • Figure out the thread priority mess
  • Assign thread affinities for a bit of a performance boost (reduce L2 cache misses)

GUIs / Formspecs

  • New main menu, which promotes Minetest better 6733
  • Refactor formspec / GUI code to be independent from serialisation 9358
  • Add new GUI API, probably based on a Lua DSL 6527
  • New GUI default theme 9802

Other

  • Make a library for reading a Minetest map without worrying about the database used. (libmtmap?)
  • Add block tinting (grass, water, sky, etc.) (for biomes)
  • Add Fullbright option

Big, long term goals (Client)

Client-Side Scripting (CSM)

See original plans: Client_scripting_plans

  • Server-Sent Client-Side Modding (SSCSM)
  • Client-side prediction APIs
  • GUI APIs

Client-Side Biomes

The server should transfer biome definitions to the client which calculates biomes locally. Crossing over into a new biome should trigger a client-side script callback Biomes have effects: 

* Full scene post-effects (tint everything a different color)
* Temporarily override settings (cloud_height, enable_clouds, water_wave_height, water_wave_length, water_wave_speed)

Hardware Lighting for Light Sources

Description
Benefits:

  • would allow variable light range and cutoff
  • would free up param1 of MapNodes to be used for storing more immediate node metadata (always a good thing!).
  • would allow colored light sources

Of course, can use dynamic lights, except they're very intensive and generally a scarce resource Dynamically generated lightmaps seem like a good way forward.
http://joshbeam.com/articles/dynamic_lightmaps_in_opengl/
RealBadAngel is currently working on shader-based lighting. What strategy is he using?

Occlusion Culling/Occlusion Query

Irrlicht does support occlusion culling. If enabled, it culls scene nodes which are entirely out of the camera's view fulcrum.
Problem: The entire map is an Irrlicht scene node. Why? What was the rationale for this decision?
Solution: Make scene nodes per MapBlock.
While we're at it, playing around with scene nodes, it'd probably be a good idea to maintain two (or three) separate nodes for each MapBlock, one for each type of material. Opaque, transparent, and liquid drawtypes. This fixes the still unsolved material transparency problem.

Big, long term goals (Server)

Voxel Area Entities

Description
How cool would it be to build a full-sized boat in Minetest, and then drive it around the ocean? That's one example of something that would be possible with this feature.
Voxel Area Entities are moveable ActiveObjects whose mesh and texture is user-modifiable via setting MapNodes in it. This is an extremely powerful feature, and while perhaps not of extreme difficulty to implement, there is very much to be done.
What needs to be done

  • A new derived SAO and CAO (they would need to be treated fundamentally differently), along with setNode() and removeNode() calls.
  • A derived VoxelManipulator class, perhaps called ObjectVoxelManipulator, which will be used to manipulate them.
  • These types of objects have meshes, and will need to be updated in MeshUpdateThread along with MapBlocks.
  • Some way to store these in the map. Perhaps these objects will get special non-block-ID keys in the map DB.
  • A decent way to serialize these objects. This detail in particular will require special attention.
  • For the previous two points, see http://irc.minetest.net/minetest-dev/2013-10-21#i_3383926 for a discussion of a possible way to handle the objects.


Envlock Scope Reduction

Description
Envlock needs to be put on a diet. Seriously. This is believed to be the primary reason for perceived unresponsiveness in Minetest - solving this would help matters immensely.
What needs to be done

  • Add a thin abstraction layer for atomic operation intrinsics.
  • Replace Environment operations that require concurrency but are simple enough with the said atomic operations and clever ordering.
  • Add a thread-safe hashtable structure to be used instead of std::map (or core::map) where locks would be needed, make use of this.
  • RCU for Map things - big structures such as MapBlocks could benefit from Read-Copy-Update for synchronization. This is the biggest change, but the reward would be immense.


Pre-generate world

Description
What needs to be done

Add colorlike to node param types

Description
Add a field like facedir, wallmounted, and liquidlike called colorlike that will allow the user to register a set of colors so that the color of some base node is modified on draw. This would save many node definitions.
This field would be (part of, at least) param2 in a MapNode. If colorlike is specified on its own, then it can define and use up to 256 colors. If used with facedir or wallmounted, then, it can only define and use 8 colors.
What needs to be done

  • Store a color LUT with the ContentFeatures for the defined node that is to be passed along from Lua when registering the node.
  • Figure out how to draw these colors in inventory screens and what not without having to prerender more textures
  • Bump ContentFeatures version


Rewrite falling code to C++

Description
nodeupdate() now very slow and can't handle large amount of falling nodes. 100-1000 falling nodes now makes server very slow, 10000+ cause segfault. To test - make sand floating island via mapgen, touch it
What needs to be done
Rewrite in core with queue (like Map::transformLiquids)

Pathfinder

See Pathfinder wishlist.

Big protocol changes

There are several features that would probably bring benefits in the long term, but can't currently be merged because of network protocol incompatibility. That is, they either can't be implemented in a way that lets old clients connect to new servers and new clients to old servers, or it would be a major pain to do so. The point of this section is to list such features so that they can be merged at once: a one-time incompatibility, perhaps coinciding with the release of 0.5.0, is better than breaking compatibility all the time. Note: Not all of the things listed below need to be done, these are just ideas.

  • A binary key-value storage which easily supports missing fields with default values (e.g. header, source, usage)
  • Using TCP for reliable data transfer (e.g. tcp_connection)
  • Make the client request mapblocks (e.g. tcp_blocks_2)
  • Improved authentication and encryption
  • Remove old compatibility kludges.
  • Many command IDs have become obsolete in the protocol: they could be consolidated
  • Change builtin CONTENT_* to 0, 1, 2


See also

Retrieved from "http://dev.minetest.net/index.php?title=TODO&oldid=6234"