Yup, that is why it is better to cite everything. Credit where credit is due and properly license everything.
I've been trying to dabble in pixel art myself but I'm just not an artist so it is slow going. I'm glad this debate started on here, because if anything it has convinced me to not take shortcuts and to simply hire others to design the art. AI is definitely an enticing shortcut but it is irresponsible. And, while the courts might be favorable to it, I think in the end it is better to support artists.
Hopefully, it becomes a safe tool at some point that artists can use to enhance their work and not something to compete with.
Yes, the exception in that video is when he generates a new fireball.. That is using their SD model to generate art, so avoid clicking that feature. But, if you hand drew the fireball to touch it up instead then the rotation and animation posing tools themselves are safe.
I think things get a bit more vague when it comes to poses. Is transferring the walk animation from something proprietary okay. I'm unsure. I think that's a grey area. However, artists have traditionaly looked at references or even sketched over references. Prince of Persia, horse walk anims, etc. There is even the rotoscoping art style.
In the video he uses an animation as a reference from artwork a user submitted to transfer that walk onto a pirate he made so in that case they're in the clear even if that was an issue though. But, I'm uncertain if there's a line there and where it is.
For example, if you hand drew a completely new character, but used the tool to transfer the walk animation from the LPC cycle. Would you have to cite the LPC artwork in your credits. Even though none of their pixels made it into your artwork.
I think the short answer is no, but the longer answer is that common courtesy would dictate that you should at least mention it but wouldn't be required to do so.
I tried to read through everything but I might have missed something 🙏 At this point I think it's hard to claim that any AI tools do not contain some copyrighted materials. However, it's a very small percentage that is art out of the full training material. It's of course needed to make it understand some art concepts and how styles look but most of the "general knowledge" doesn't come from art. There are some new attempts at creating models trained on only open data and the first might be released in a few months time 🙂
It might be possible to create separate models eventually that are based on these open models but it's very unlikely that the latest research that we often build from will be based on open datasets. It would be far out of our reach to be able to afford pretraining all these models from scratch on only open data.
Until an open model is released, it's hard to make any claims of being free of copyrighted art and other materials. We have tried to stay away from copyrighted materials for finetuning but we decided to stray from that path with the rotation tool and include old games for the reasons Kaninen described that you're not risking doing the same thing as someone else. We're also an extremely small company (haven't gotten a salary yet from this but maybe after Christmas) and it's unclear what we should do when all the big companies with lots of VC money like Midjourney, OpenAI, Scenario, Leonardo, Google, etc. are just pushing ahead 🤷♂️
Hope that answers some of your questions 🙂
I responded:
thx for the update, i'll let oga know the current situation and its up to them if they want to make use of the tool... from what i can tell it's obvious that the rotation and animation tools generate art from your own artwork that you're inputting, and only use knowledge of perspective gained from the model to assist in performing the rotations/poses etc.. but it'll be up to the oga community to decide if that is reasonable/fair/legal for them to move forward or not on using the tool.. thanks for the quick response
For context for anyone who skipped to bottom, we're discussing:
tldr; The rotation and pose transfer animation tools provided by pixellab use your own artwork as input so if you're inputting open source art you're producing open source art. It is only when you use their art generation tool that things get a bit more questionable, so as long as you avoid that and only input your own art, or properly licensed art then everything works out. The reason this matters is that imagine for a moment creating a new set of animations for LPC for example, and then being able to efficiently (albeit still with some work) transfer the poses onto all of the clothing, armor, headwear, etc from the rest of the LPC collection onto that new animation seamlessly. And, likewise, taking 4-directional characters and making them 8-directional, etc. This is some next level shit. It also still requires pixel art skills, as you can see in the videos, so it's not a completely automatic transfer and is just another tool to assist artists.
The tool that I mentioned is amazing, but unfortunately they claimed their model engineer wasn't around to read Discord and then afterwards he reacted to a gif someone posted... sooo... I don't think we're going to actually get answer out of them on this.
If we base it purely on the developer's answer via Discord, which I was holding off on posting here until the model engineer could chime in, it doesn't look good though.
"Kaninen — Today at 11:25 AM We're training on a mix of paid assets, our own creations, commissioned work, and open license but we are fine-tuning from base models that have trained on the entire internet. Note: The rotation tool includes some old retro pixel art games
Kaninen — Today at 11:40 AM Where would you draw the line btw? Would fine-tuning lets say an SD model which has trained on the whole internet be unacceptable?
Kaninen — Today at 11:41 AM Rotation and animation for example will always be based on your character so if you hand made it, you will never run into an issue of it not being oc"
So, here's the question.... if an AI tool was fine tuned on proprietary artwork, to understand perspective in order to rotate pixel art for example. But, only uses that understanding in conjunction with inputs that are public domain, open source or made by yourself and so the input and output could be considered original. Is this fair and lawful? And, even if it is, does it offend your sensibilities as an artist. Which is another valid issue.
I've jumped into the discord for PixelLab to ask them what their model was trained on. I'll post an update here when I get an answer. Their tool is at:
Edit: I talked with the dev and he is contacting the model engineer and will get back to me with a definitive answer.
I also recommended they add a feature to the tool that will produce a Credits file like Universal-LPC-Spritesheet-Character-Generator does. And, I recommended they provide a way to include/exclude certain licenses so that not only is the artist using the tool aware of the licenses involved but can restrict it as needed.
I'm on the fence about AI, I like all the doors it opens but I also dislike that creators are being robbed. However, if it is trained on CC0/OGA art or even your own art in order to speed up the process of creating art then it is just another tool to help artists along. But, the end result should mention all the licenses used just as you would if you had manually created the art piece using art from others.
I know it leaves a bad taste in mouths because of the backlash it has received. But, I don't think the opengameart community should overlook the power of AI to really assist in the OGA cause. For example, imagine using it to assist in rotating your sprites. Or, helping to create new poses of the same sprite by being able to grab and drag on controlnet joints. There is really a lot of possibilities here to extend for example the LPC set.
We should sit down as a community and maybe come up with responsible AI, just like having a community tool like the LPC universal generator, a universal oga AI sprite rotator or pose tool would be really amazing. My two cents.
The science of this stuff is moving pretty fast. Here's an example of a rotator tool:
Yup, that is why it is better to cite everything. Credit where credit is due and properly license everything.
I've been trying to dabble in pixel art myself but I'm just not an artist so it is slow going. I'm glad this debate started on here, because if anything it has convinced me to not take shortcuts and to simply hire others to design the art. AI is definitely an enticing shortcut but it is irresponsible. And, while the courts might be favorable to it, I think in the end it is better to support artists.
Hopefully, it becomes a safe tool at some point that artists can use to enhance their work and not something to compete with.
Yes, the exception in that video is when he generates a new fireball.. That is using their SD model to generate art, so avoid clicking that feature. But, if you hand drew the fireball to touch it up instead then the rotation and animation posing tools themselves are safe.
I think things get a bit more vague when it comes to poses. Is transferring the walk animation from something proprietary okay. I'm unsure. I think that's a grey area. However, artists have traditionaly looked at references or even sketched over references. Prince of Persia, horse walk anims, etc. There is even the rotoscoping art style.
In the video he uses an animation as a reference from artwork a user submitted to transfer that walk onto a pirate he made so in that case they're in the clear even if that was an issue though. But, I'm uncertain if there's a line there and where it is.
For example, if you hand drew a completely new character, but used the tool to transfer the walk animation from the LPC cycle. Would you have to cite the LPC artwork in your credits. Even though none of their pixels made it into your artwork.
I think the short answer is no, but the longer answer is that common courtesy would dictate that you should at least mention it but wouldn't be required to do so.
Got an update from PixelLab's model engineer:
I responded:
For context for anyone who skipped to bottom, we're discussing:
https://www.pixellab.ai/
https://www.youtube.com/watch?v=bNtMm0SeJ74
https://www.youtube.com/watch?v=zBfVT5pwCSs
tldr; The rotation and pose transfer animation tools provided by pixellab use your own artwork as input so if you're inputting open source art you're producing open source art. It is only when you use their art generation tool that things get a bit more questionable, so as long as you avoid that and only input your own art, or properly licensed art then everything works out. The reason this matters is that imagine for a moment creating a new set of animations for LPC for example, and then being able to efficiently (albeit still with some work) transfer the poses onto all of the clothing, armor, headwear, etc from the rest of the LPC collection onto that new animation seamlessly. And, likewise, taking 4-directional characters and making them 8-directional, etc. This is some next level shit. It also still requires pixel art skills, as you can see in the videos, so it's not a completely automatic transfer and is just another tool to assist artists.
Yea, this all jives with my understanding.
The tool that I mentioned is amazing, but unfortunately they claimed their model engineer wasn't around to read Discord and then afterwards he reacted to a gif someone posted... sooo... I don't think we're going to actually get answer out of them on this.
If we base it purely on the developer's answer via Discord, which I was holding off on posting here until the model engineer could chime in, it doesn't look good though.
"Kaninen — Today at 11:25 AM We're training on a mix of paid assets, our own creations, commissioned work, and open license but we are fine-tuning from base models that have trained on the entire internet. Note: The rotation tool includes some old retro pixel art games
Kaninen — Today at 11:40 AM Where would you draw the line btw? Would fine-tuning lets say an SD model which has trained on the whole internet be unacceptable?
Kaninen — Today at 11:41 AM Rotation and animation for example will always be based on your character so if you hand made it, you will never run into an issue of it not being oc"
So, here's the question.... if an AI tool was fine tuned on proprietary artwork, to understand perspective in order to rotate pixel art for example. But, only uses that understanding in conjunction with inputs that are public domain, open source or made by yourself and so the input and output could be considered original. Is this fair and lawful? And, even if it is, does it offend your sensibilities as an artist. Which is another valid issue.
I've jumped into the discord for PixelLab to ask them what their model was trained on. I'll post an update here when I get an answer. Their tool is at:
https://www.pixellab.ai/
Edit: I talked with the dev and he is contacting the model engineer and will get back to me with a definitive answer.
I also recommended they add a feature to the tool that will produce a Credits file like Universal-LPC-Spritesheet-Character-Generator does. And, I recommended they provide a way to include/exclude certain licenses so that not only is the artist using the tool aware of the licenses involved but can restrict it as needed.
We'll see what they come up with.
I'm on the fence about AI, I like all the doors it opens but I also dislike that creators are being robbed. However, if it is trained on CC0/OGA art or even your own art in order to speed up the process of creating art then it is just another tool to help artists along. But, the end result should mention all the licenses used just as you would if you had manually created the art piece using art from others.
I know it leaves a bad taste in mouths because of the backlash it has received. But, I don't think the opengameart community should overlook the power of AI to really assist in the OGA cause. For example, imagine using it to assist in rotating your sprites. Or, helping to create new poses of the same sprite by being able to grab and drag on controlnet joints. There is really a lot of possibilities here to extend for example the LPC set.
We should sit down as a community and maybe come up with responsible AI, just like having a community tool like the LPC universal generator, a universal oga AI sprite rotator or pose tool would be really amazing. My two cents.
The science of this stuff is moving pretty fast. Here's an example of a rotator tool:
https://www.youtube.com/watch?v=bNtMm0SeJ74
And, using a skeleton:
https://www.youtube.com/watch?v=zBfVT5pwCSs