$12256 / $11500
Normally I'd post this in the OGA2 forum, but in this case I want to get opinions from a broad cross-section of our users, so I'm posting it here instead.
OGA2 is going to have something akin to a rating system for art. I use the term "rating" loosely because it's really more about completeness and usability than an actual rating, but it fills the same niche. I propose dividing art into three main rating categories: WIP, Complete, and Refined. In addition, there will be two more possible ratings: Unrated and Abandoned, which I'll get to in a minute. Here's a description of each rating:
In addition, a piece of art may be listed as "Unrated", which just means that no editors have assigned a rating to it yet. Finally, the "Abandoned" rating denotes a piece of art that was submitted at the WIP level and has not been touched by the author for a long time. Modifying an abandoned work will automatically switch it back to WIP. It should be noted that a work which is completely inappropriate for a video game (pure advertising, etc), will simply not be published, and will not appear in the archive. "Here's a screenshot of a thing that you can come to my site and download" isn't an incomplete submission -- it's in bad faith. (On the other hand, "Here's this asset, you can come to my website to download more like it" is perfectly fine.)
At any rate, the way I imagine this will work is that, when a piece of art is submitted, a site editor will assign a rating to it. If the art is rated WIP, the editor will make some suggestions to bring the art up to the "Complete" level (which may be anything from minor corrections to a major overhaul). Optionally, the editor will be strongly encouraged to suggest ways that the piece may be brought up to a Refined rating. The difference between a Complete and Refined work can be many hours, but submitting refined works will earn you special medals above and beyond those for submitting WIP and Complete works.
By default, an art search will only return Complete and Refined works; however it will be easy for the user to expand or limit their search to include other ratings as well. Furthermore, it might be interesting to have a WIP queue that highlights the most actively maintained WIP pieces.
Ultimately, the idea here is to promote art (much like software) as a collaborative process, as opposed to something you just create once and never update. As such, OGA's rating system isn't "bad" to "good", although it can be considered roughly analogous to that on some levels. The difference is that the rating isn't final; instead, it suggests a path that a work can take to completion and eventual refinement.
At any rate, I'd be interested to hear everyone's comments and suggestions. This is still (to some extent) open to discussion, although at this point I'm looking for refinements to the idea as opposed to suggestions for a complete overhaul. :)
Peace,
Bart
This website's content is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
I was quite excited when this general idea was first brought up. I feel like this would be more useful, and does really have the potential to encourage the sort of collaboration which could add a lot of value to the site. I have, for instance, had a lot of fun working with Blarumyrran's and Antifarea's works, and it would be neat for that sort of thing to be encouraged.
I do wonder, though, whether there will come a time when hand-assigning art will become logistically difficult (though the results would definitely be better).
I guess it will be useful for reviewers if get a list of checkboxes on reasons why the art is considered WIP, along with a comment field. The reason is that a few reasons (per art type) are applicable to a lot of "WIP" art, and in case the reviewer is bored to write the same reason yet again, he might appear rude to the artist, which won't happen with checkboxes.
Am I making any sense?
Seems like a good idea, particularly the check boxes. I would suggest that being able to filter WIP results by the level of completeness during a search might also be handy.
I have been think about this and think we not giving ourselves a enough slack with only three categories of finished-ness.
I think a better solution is to a 0 to 100 scale where 0 is a draft concept, 50 is a workable draft, 75 is a high quality piece, 100 is a refined production ready piece. every piece starts at 100. User then subtract points from and must supply 1 reason for every 5 pts subtracted. Admin will have more freedom in point subtraction but must always provide technical reasons. subtracted numbers are averaged or some other way.
Generally, I think it's better that new submissions not start out on top--it sorta leads to chaos, and makes the ratings less meaningful. I also don't think it's a good idea to make it a difficult process, because that will lead to people simply not bothering, thus perpetuating bad pieces staying on top. At best, you'd probably end up with a red-steckled elbermung sort of situation, where it just devolves into pointing at things and saying "No, no, no."
not pictured: excellent and meaningful commentary
My biggest fear with that is that in order to keep things easy, we need to able to be at least somewhat arbitrary. The issue there is that someone will eventually get offended that we used varying standards to apply a rating, but to some extent you need to hold on to that level of agility. Take Wikipedia, for instance. Their notability standards are so utterly convoluted now that they can be interpreted any way people want to, which ultimately leads to the same problem they were intended to prevent. Keeping the standards simple but a little bit open to interpretation (and being willing to consider changing a rating if someone has a good objection) might be the best way to go.
Starting at 100 is a way of saying we think your contribution is important. Like when a teacher tells you your grade starts at an A+ all you have to do is keep it. I don't think ratings should be arbitary at all. They should be based on basics like form, line, color, texture, space, value and sound art principles.
I like the idea of degrees of completeness rather than scales
For moderating make a section where long time contributors can give suggestions to new works if we outgrow the phase where moderators can't handle them
However I do suggest another degree of completeness as "part" in which it can be use with other models or used as a derivate other models and don't quite deserve the title of WIP or Abandoned even if it currently can't be use in games
Yeah, i like this idea, go on bart =)
I think idea with "rating" is right, but in my opinion "rating" should be determined by community voting and not by single person: moderator or someone else.
I imagine the whole process to be something like this:
1. After submission, work goes to "New works" section. Hangs there for a while, a week for example, accumulating "rating score" from users and suggestions about improvements. Then it is sorted according to it's current "rating", i.e. moves to "Complete" or "WIP" section or somewhere else.
2. When some updates are made to this work, it goes to "Updated works" section, where users may vote again. And after some time it is sorted according to it's new rating.
3. And so on...
For all of this to work better, it would be nice to have some kind of "tickets" system. I.e. instead of writing his suggestion in comments, one can create a ticket with description of his idea and then attach it to work with his username.
When updates are made, author marks tickets related to update so every user can easily see, whether his ticket was closed and should he bother revoting for work in "Updated works" section.
Also some thoughts about "refined" status. I think that work may be perfectly complete but far for being "refined" and, in some rare cases, vice versa. That's why it is better to create a separate vote for it with only "yes" and "no" variants. If number of votes becomes bigger then X, 50 for example, and ratio between "yes"/"no" bigger then Y, 3:1 maybe, then work becomes "refined". In order to encourage voting some list of nominee should be generated periodically.
As for actual rating system, i really don't sure which is better. On the one hand, "WIP"/"Complete" gradation is enough to, well, distinguish complete works from work-in-progress works and encourage improvements. On the other hand, even complete works may need some minor corrections and that's where more complicated rating system becomes handy.
P.S.: Sorry for my english. I hope that meaning is more or less clear.
>Anon
I like your idea,the best inspiration would be the bug system is since it kinda works the same
Maybe the ticket system would have a severty value which will show how important it is to be "fixed" to achive the next level while the other are considered refinements?
I like the original idea, I just have some questions about the editors:
I love anon's bug tracker idea, infact there is a drupal module that could be commendeered for this task :)
http://drupalmodules.com/module/project-issue-tracking
The ticket idea sounds great to me. I always like to have feedback on specific things that should be changed to make my work better, or whatever a dev needs to put it in their game. It could also be a good way of requesting specifications for their game, like whether or not they need a bump map, what texture sizes to use, etc.
Wow, lots of good responses here. Robin asked some good questions, so I'll try and address those first, starting with:
How many editors concern themselves with a single art piece?
Preferably not too many. As an art submitter, it's important that there be some kind of consensus. For instance, if I submit a piece of art that's considered WIP, then I'm going to want a firm list of things about it that need to be fixed before it can be considered complete. If there are too many editors, there will almost certainly be a lot of differing opinions about each art piece, which will make the whole review and improvement process annoyingly cumbersome on both sides.
Here's a thought: When a piece is submitted, it doesn't get an official rating until three editors have rated it (with required comments if they rate it WIP). The editor ratings will be blind by default, so as an editor you won't be influenced by the rating another editor gave the work, because you won't be able to see it until all three ratings are in. Once three editors have rated it, the rating will be assigned based on the editor votes, as follows:
Once three ratings are in, the submitter will be able to see the editor's ratings and comments. When they improve their work, the editors will be notified, so they can change their rating based on the improvements. If the editors don't respond to new changes within a few days, the work will open up to other editors to respond to those changes and alter the rating.
How many editors would there be? (In absolute numbers or % of the total community.)
At this point, I have no idea. This is new territory, so the number of editors we have will be based on need.
How are the editors selected? (Hand-picked by bart on basis of merit, elected by the community, chosen amongst the existing editors...)
The first editors will be hand picked. Beyond that, I'm entertaining a few possibilities, my favorites being:
What would the turn-over rate of editors be? (Selected for an x amount of time, appointed for "life"...)
I had assumed it would be a lifetime appointment, although it would have to depend, I think, on them remaining active. I don't want the editor list to get too bogged down with inactive editors.
Finally, I need to point out the difference between editors and administrators. Admins will have editor privileges, but as we grow and take on new editors, I'm hoping that the admins will need to do less and less editing. Ideally, it would be nice if content rating determinations could stay "grassroots" in a sense, wherein people who do the bulk of the editing work got there by community merit and not a personal appointment. I'd like to avoid ending up with the sort of cliquish management that plagues sites such as Wikipedia, and this means giving the community at large the power to gain trusted user status.
Administrators will, for the record, always be hand picked by me, just for the safety of the site.
It might be nice to have a feature where you can submit a work as WiP from the start. That would save editing pieces which were never meant to be considered complete in the first place.