Major Improvements In Development Come With Cocos Creator 3.6

Starting with the integration of 2D and 3D in 3.0, the developers have been with us through six significant releases. These include additional platform support, such as being the first in the world for HarmonyOS support and the first in China with Nintendo Switch support, optimization of the lighting models and support for mobile deferred rendering pipeline, the addition of essential 3D features such as the Marionette animation system, and improvement of the development experience and performance optimization for 3D and 2D projects. This series of release evolutions over the past two years fully illustrates the attitude that small steps can lead to a thousand miles.

Within the Cocos team, we have been treating 3.6 as the most important milestone since the 3.0 merge. It has had the most extended iteration cycle to date and has significantly evolved in development efficiency, runtime performance and graphical expressiveness.

Since the release of 3.3 last year, some of the most critical development team members have been working on the 3.6 branch. Such a long development circle is often necessary for some underlying fundamental refactoring. The specificity of a commercial engine serving the entire industry dictates that it must constantly keep up with hardware development to challenge for the best graphical performance and continuously improve productivity on the production side to maintain compatibility with projects and scalable adaptations to different hardware environments.

These challenges mean that the engine requires continuous iteration and refactoring of the underlying framework, sometimes disruptively, such as evolving from a 2D engine to a 3D engine. Most of the time, this iteration is continuous, such as our original progress towards 3.6. This advancement has a lot to share, and we’d love to share some of the epic highlights of this newest version of Cocos Creator. We can see the eternal proposition of engine development from our work: Continuous self-change to adapt to hardware updates and changes in user needs.

Note: Windows platform no longer supports publishing to Win32. Only Win64 is supported. Minimum version support for the iOS platform changed from iOS 10.0 to iOS 11.0 (in order to use C++ 17).

For a complete list of all the updates and the change log, please check out the developer notes in our forum.

Higher Performance with More Native Modules

For higher performance, better compatibility, more scalability, and better productivity, the Cocos Engine team has been iterating and refactoring the core engine framework continuously since the release of version 3.3.

It took the engine team more than a year to refactor the engine in multiple phases and modules to achieve a significant all-around improvement while still maintaining compatibility with old projects.

While bringing significant performance improvements and rendering enhancements, it also lays the groundwork for further iterations of the engine in the future.

These native modules also mark a significant milestone by achieving the same level of 2D rendering performance as 2.x. This means that 2.x users can upgrade to 3.x with confidence if they have to add 3D or have ongoing iterative needs. The 2D batching and rendering submission processes are all done natively to achieve 2.x-like performance. Of course, there are still some leftovers, such as Spine batching support, but this means that there is a higher ceiling for 3.x 2D rendering performance on top of the native one.

Materials, Animation, and Particle Upgrades

In terms of evolution, 3.6 brings a new look, a new material API, a new import experience, and new features such as an animation embedding player and particle noise maps. What does all this mean from a user perspective? First is the evolution of the editor’s UI and interaction experience. Hundreds of detailed changes will make developers intuitively feel the editors give them a better experience and more comfortable appearance. Secondly, a series of new features in the lighting model and rendering algorithm will allow Cocos to develop games with graphics comparable to the quality of modern mobile games, especially the cascading shadows and anisotropic materials shown below, which will definitely make your games shine.

Detailed Key Updates

Surface Shader Custom Materials

Surface Shader uses a unified rendering process and structure that allows users to create surface material information in clean code, specifying the lighting and shading model used for the combination. The advantages over the older version (Legacy Shader) are easier to write and maintain, better version compatibility, and less prone to rendering errors. And you can get a lot of public features from the unified process, such as unified full-scene lighting and Debug View debugging capabilities.

Cocos Creator also makes it easier to extend a variety of common and complex materials to users and will support Shader Graph to automatically generate Effect code, which can significantly improve the efficiency of Shader developers.

New Rendering Debug View

This feature provides a variety of custom display modes to help users investigate issues of material, lighting, and shadow display much faster. It also allows users to view specific scene information more clearly and provides a reference basis for optimization decisions.

Cascaded Shadow Maps

The regular shadow map has a serious drawback: when the projection area is large, a very high-resolution shadow map texture is needed to achieve a good shadow effect, which may exceed the limits of the hardware capacity. If the resolution is not increased, the shadows will be jagged, shapeless and unclear, but reducing the projection area will result in a very short shadow distance, objects a little further away will lose their shadows. Adjusting the balance between shadow distance and shadow quality is a major headache and it has been in previous versions.

CSM( Cascade Shadow Map ) divides the view frustum into multiple blocks from near to far, with a smaller projection area for the near block and a larger projection area for the far block, which is equivalent to an adaptive shadow map, thus increasing the utilization of shadow map texture several times. It can have a very precise shadow for closer objects and a just fine shadow effect at a longer distance from the view point, so developers no longer have to spend their time on shadow parameter settings.

GGX Convolution

Since the specular of direct lighting are using the GGX BRDF model, for environment lighting, the same BRDF must be used for spherical convolution in order to make the lighting effects of both light sources correspond. In addition, by storing the reflection information of different roughness in the corresponding mipmap level, the convolution calculation can fix the following problems compared to the automatically generated mip maps:

1. Flooding and trailing of environment lighting specular are greatly reduced
2. Different effect from Substance’s standard PBR material
3. Difference between specular of direct lighting and environment lighting

As shown in the figure:

Anisotropic Lighting

With the benefit of Surface Shader, we can fully extend the PBR lighting model. In v3.6 there is complete support for isotropic and anisotropic materials and lighting models of direct lighting and environment lighting. You can work with the Substance PBR material library to create brushed metal, hair, silk, etc.

Editor UI is completely upgraded

V3.6 has a brand new editor UI. This revision is based on a “more coordinated” visual system, “more intuitive” visual feedback, and “more immersive” interaction feeling, and a full-scale reorganization of UI and interaction. In the future, we will continue to standardize design based on Cocos’ design goals, design system, and design principles and update and iterate to optimize core interactions and workflow.

New preview mode

Development efficiency is a core value that Cocos Creator places great importance on, and in v3.6, this has been further improved. In addition to the “Preview in Browser” and the “Preview in Simulator”, developers can now use the “Preview in Editor” to preview the project. In this mode, the scene manager will run the code directly, allowing developers to debug scenes in real-time. While bringing a seamless preview experience, it also makes up for the shortcomings in debugging.

This feature is currently in an experimental stage. We welcome you to give us more feedback about it. In the future, we will also continue to focus on development efficiency and continue to improve user experience in script compilation, project debugging, and build release

Embedded Players for Animation Clips

Animation Editor adds the ability to embed players. Developers can embed other particles and other animations in any animation. Moreover, it can be programmed in a way like editing tracks in video editing software, with free adjustment of duration and playback position.

After editing, the content of the embedded player will play along with this AnimationClip, which is supported in both the Animation component and Animation Graph. This feature is also available for animations from FBX, allowing for more flexible effects control and solving the problem of imported animations being difficult to re-edit.

Both particle and animation players are currently available in 3.6 and can be enabled in the lab settings.

Localization Editor with Integrated Multilingual Support

As the market changes, the demand of cocos users for localization increases significantly, and there is a lack of a powerful L10n tool, so the official Localization Editor plugin is now built-in for version 3.6.

Its core purpose is to improve the efficiency of user translation through a highly automated tool and to lower the threshold of user use in a codeless way, to reach anyone who can use the tool out of the box.

The core functions it contains are:

1. support machine translation, currently connected to Google, Youdao translation

2. support for one-click extraction of the content to be translated

3. support .xls, .xlsx, .csv, .po file batch import and export

4. support multi-language fast switching and preview

Scene Editor

New surface snapping and vertex snapping capabilities for the scene editor

• Surface Snapping (Press and hold ctrl/cmd + shift)

• Vertex Snapping (Press and hold v)

Support box selection function, you can select multiple objects at once

Marionette Animation System Updates

– Animation Graph supports previews while editing.

When editing the animation state machine, you can preview it in real-time to see the result of transition and blending, and debug it quickly without having to run the game after each modification to see the result.

– Animation diagram supports variable and layer renaming

– The animation map supports transition line ordering.

– The “End Start Time” property has been added to the animation diagram to allow the target animation of a transition to start playing from the specified position.

– The animation diagram can now configure a transition as interruptible to allow the specified transitions to be interrupted by other transitions. This feature can be enabled in the lab settings.

Particle System Updates

– A new Noise module, which brings a more natural and controllable random motion effect for particles.

– Models in particle systems support GPU Instancing to improve emitter performance

– Particle editing supports “Undo” operation, improving the efficiency of particle effects creation

– Support for previewing and controlling particle groups in the Hierarchy

No splitting of models by default

Due to the limitation of the number of uniforms, in CPU computed skeletal animation, when the number of skeletons exceeds a certain value, it is not possible to store all the skeleton data by uniforms at once, so in the previous version, we may split the meshes and bones. We often received feedback that one of our character models would take up multiple draw calls, mainly because it was automatically split here. What’s more is that since splitting cannot be done at runtime and can only be pre-processed, our splitting criteria can only be used as a reference for lower hardware devices and drivers (iPhone 6 WebGL), which is very limited for the number of skeletons.

Therefore, Cocos Creator 3.6 has made a strategic optimization of this issue as follows:

• By default, the model is no longer split, and no changes are made to the imported model data (also keep model settings unchanged)
• If the number of bones does not exceed the limit of the actual runtime driver, it will be passed directly using the uniforms
• If the number of bones exceeds the limit, it will be passed using textures

Note: Using textures to pass skeletal animation data requires the ability to access textures in the vertex shader. This feature is minimally supported by OpenGL ES 3.0, WebGL 2.0. But relies on GL extensions, it has almost 100% coverage on devices which only support OpenGL ES 2.0 and WebGL 1.0 , so there is no need to worry about compatibility issues.

This option is currently retained only to maintain compatibility with old projects and will be considered for removal when appropriate.

Mesh updates

– Support for dynamic mesh, developers can now update the mesh data at runtime through API

– Add new curve type Spline to GeometryRenderer (support Linear, Bezier, Catmull-Rom curves)

Support for terrain height brush