I was looking a long time for a policy which binds a user to a bucket. The docs are not very helpful for beginners, but I talked with an advanced user and he said it is okay to share his solution, since he is not on Lemmy

Create a new policy and fill this in:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::${aws:username}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::${aws:username}/*"
            ]
        }
    ]
}

If you now create a user, just assign the user to only this policy, nothing more. The user is now allowed to create a bucket with the same name as the users. So a user Alex can only create the bucket alex and has complete access to it. The user won't see other users buckets.

All credits belongs to the very helpful person, not me ☝🏻☺️