feat(RotationSystem): Deep Learning Support

[1zun4]

DeepLearning Rotation System

The successor to #5642 - which was far from finished, and unfortunately people tried to use it before I had it ready, so they asked for support and gave feedback on an unfinished pull request.

I am now looking for people who can play and record their combat data - in order to have enough for a ready to use model. I have prepared three models myself within 20 minutes total.

Features

• Deep Learning Framework
• Download Progressbar
• Rotation Model
• Model Managment Command
• Training Recorder
• Combat Recorder
• Base Model

For the future

• Curve Recorder
• Model Configuration
• Parameter Configuration
• Model Metadata
• Model Load On Select

Data Collection

Minarai Combat (PTB)

No Recording. Imagine playing Practice, over... and over... and over...

Minarai Trainer (PTP)

2025-02-21.00-11-21.mp4

Model Managment

• .models create <name>
• .models improve <name> (!WARNING! This will not filter previously used data, remove it before improving!)
• .models delete <name>

Roation Configuration

In case our model is not able to keep up with high movement, we can use Correction which will act as Aim Assist for the Model. This is very useful as e.g. when only training with PTB-Mode we end up with a lot of low-delta data causing our model to move too slowly. This is NOT an issue with PTP-Mode though. We can also amplify the model output, which has a similar effect and makes it much faster, but it will often aim in the wrong places or start to shake as it tries to correct the wrong output on the next tick.

Demo (PTP+PTB Model)

Model and Data

Minarai-Izuna-20min-Training.zip

Video

2025-02-21.01-13-06.1.mp4

In-Depth

As we can see, it sort of combines the concept of Acceleration as well as LazyFlick (known in Legacy). @mems01 has already done a good job of replicating this behaviour in traditional methods, and I still think it's probably the better solution compared to Deep Learning - but it has to rely on the developer to keep upgrading, updating and maintaining the curves, because as @xkeksi has already pointed out, they always have recognisable patterns. Deep learning is a pattern in itself, but it's a very large and dynamic pattern, and with enough data we can make it behave differently - and with just a little bit more data we can build a whole new pattern without writing a single bit of code.

Explanation

The problem with this model is that we're relying on the LiquidBounce Aim Point Tracker - which will conflict with our PTB model, which wants to introduce its own point tracking. That's why it started to drift a bit - but overall this clip shows how it works, and from the current feedback I'm getting, it seems I've finally got it right.

Feedback

Thank you very much. It's always nice to hear that something you've been working on for the last 5 days is working. Of course we still have to see how it works with anti-cheats, but I will leave that to the config creators and just provide an easy to use model system :)

To reviewers

Do NOT submit any code feedback. Thank you.

[twcult]

Hurttime may be superfluous in this case, but I could be wrong. Look at its priority in features

[1zun4]

Hurttime may be superfluous in this case, but I could be wrong. Look at its priority in features

The data is being recorded - however, it's not passed into the model.

LiquidBounce/src/main/kotlin/net/ccbluex/liquidbounce/deeplearn/data/TrainingData.kt

Lines 69 to 90 in e445ca9

	val asInput: FloatArray
	get() = floatArrayOf(
	// Total Delta
	totalDelta.deltaYaw,
	totalDelta.deltaPitch,
	// Velocity Delta
	previousVelocityDelta.deltaYaw,
	previousVelocityDelta.deltaPitch,
	// Speed
	targetDiff.horizontalLength().toFloat() + playerDiff.horizontalLength().toFloat(),
	// Distance
	distance.toFloat()
	)
	val asOutput
	get() = floatArrayOf(
	velocityDelta.x,
	velocityDelta.y
	)

[sqlerrorthing]

@1zun4 do u know russian? (at least a little?)

[vanvac]

dam ai aimbot is cazy (anticheats are cooked)

[sqlerrorthing]

invalid char in progress bar (after percents and before progress bar bar)

[1zun4]

invalid char in progress bar (after percents and before progress bar bar)

That is normal. It's a line break and only works using the LiquidBounce HUD.

[sqlerrorthing]

лол хфывфхывхыфвх

invalid char in progress bar (after percents and before progress bar bar)

That is normal. It's a line break and only works using the LiquidBounce HUD.

check for it?

[1zun4]

check for it?

Not relevant.

[superblaubeere27]

[sqlerrorthing]

do not review a draft pr

[commandblock2]

Holy *, I am speechless, never thought a tiny model like this could be perfect for this task

private fun createMlpBlock(outputs: Long) = SequentialBlock()
    .add(Linear.builder()
        .setUnits(128)
        .build())
    .add(Blocks.batchFlattenBlock())
    .add(BatchNorm.builder().build())
    .add(Activation.reluBlock())

    .add(Linear.builder()
        .setUnits(64)
        .build())
    .add(Blocks.batchFlattenBlock())
    .add(BatchNorm.builder().build())
    .add(Activation.reluBlock())

    .add(Linear.builder()
        .setUnits(32)
        .build())
    .add(Blocks.batchFlattenBlock())
    .add(BatchNorm.builder().build())
    .add(Activation.reluBlock())

    .add(Linear.builder()
        .setUnits(outputs)
        .build())

You did make it happen and that's crazy. I think this is the exactly right step to go.
I am indeed thinking about a neural network in the game tho I would do it with the client sending gamedata to / receiving input from another process (python or whatever).

[sqlerrorthing]

FUCK
AN HOUR OF RECORDING THIS SHIT

[sqlerrorthing]

@1zun4, record with 100k samples (again, but it was saved successfully this time.)
download record (55.4 MiB), trained model (45.5 KiB)

[MukjepScarlet]

@sqlerrorthing we probably need a more effecient way to save the record XD

[MukjepScarlet]

Well I found a eaiser way to solve the save issue

[1zun4]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Thank you. That is with PTP Training (Minarai Trainer)?

[sqlerrorthing]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Thank you. That is with PTP Training (Minarai Trainer)?

yes

[MukjepScarlet]

It will be good if we can share and get models on the incoming marketplace, I'm looking forward to it.

[vanvac]

will this feature require a decent computer to run cause from what im seeing its a ai controlling the player and ai dont seem too easy to run on not good pc's

[sqlerrorthing]

will this feature require a decent computer to run cause from what im seeing its a ai controlling the player and ai dont seem too easy to run on not good pc's

i think not. maybe add a setting, like a toggle switch.
remember that ai doesn't need many neurons to work

[liquidsquid1]

this seems really interesting! i can do a 6 hour recording soon and upload the model here with minarai combat? ill just be playing pika xd

[DrParanoya]

It looks way too legit

[vanvac]

is this planned to release with lb 1.27.0 or its done when its done type of thing

[1zun4]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Next time please upload to GitHub instead. Also compress the JSON before uploading.

[sqlerrorthing]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Next time please upload to GitHub instead. Also compress the JSON before uploading.

oh, but this file more than 10 MiB

[1zun4]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Next time please upload to GitHub instead. Also compress the JSON before uploading.

oh, but this file more than 10 MiB

Not as ZIP.
1yc28l82.zip (4,1 MiB)

[MukjepScarlet]

We can have a special file format for List<TrainingData>. But this will make the files unreadable.

[sqlerrorthing]

@1zun4, record with 100k samples (again, but it was saved successfully this time.) download record (55.4 MiB), trained model (45.5 KiB)

Next time please upload to GitHub instead. Also compress the JSON before uploading.

oh, but this file more than 10 MiB

Not as ZIP.
1yc28l82.zip (4,1 MiB)

не подумал, сорянчик