On OpenGameArt & what I see as problems
On OpenGameArt & what I see as problems
_The problem with ratings in projects relying upon goodwill_
First of all, ratings are good and should be integrated wildly. A browsing person looking for images to use in his game or just to look at, will benefit massively from ratings; without being aided by rankings, he's likely to never reach the sort of image he wished for at all even if it existed in some corner of OGA (as there are, of course, way too many images on OpenGameArt to simply browse through them all - or even some large subset of all as responding to some more popular search terms - and in time, their number will only grow so the importance of ratings will increase).
To the problem. This is something I also observed in Musopen.org (which is an excellent project, mind you - with a very nice, sleek, easy-to-use website as well) - entry ratings don't work well in projects relying upon goodwill. From the perspective of the submitter, any entry is an act of goodwill, with a price-per-value ratio of 0. From the perspective of the site owner, most entries have a very low price-per-value rating (if the entry is of low enough quality, it devalues the whole website slightly, and hinders browsing because it stands on the site's browser's way towards better entries - but such entries are exceptions rather than the rule, so the perspectives don't differ much). If a person gives a rating to an entry, and measures the rating by how good an act was it from the submitter to submit this entry, which sounds unintuitive when stated out this way but is really an intuitive thing to do, he will give the entry a very high rating. So, he will give all entries high to very high ratings on a scale unrelated to an objective scale - eg a scale that compares them to the higher commercial standards, or even a scale that compares them to the best pieces on the site! The ratings on Musopen.org are absolutely meaningless - even the few low-to-medium ratings are largely meaningless - I don't know if they're given by different groups of people who like to give different grades, but the ratings are certainly very disconnected with the actual quality of the piece. Instead, it should be up to the administrator to be "unempathic" and objective, and to either give ratings himself only, or to somehow enforce that all raters will be unempathic and hence objective.
OGA doesn't have such ratings, so the problem doesn't stand out as far. Which is bad, because as said, ratings are good. However OGA has:
1) "Featured Art"
2) Number of people who named the entry a "favourite".
The "favorites" count lacks on the following points:
1) it has only 2 levels of quality; something either is a favorite or it isn't
2) a "favorite" is bound to be even more subjective than a quantified rating already by the virtue of its name
3) "favoriting" a piece can be seen as an act of appreciation of the artist, rather than the entry itself
4) medals encourage people to "favorite" things, which again makes it unobjective.
The "Featured Art" group lacks on the following points:
1) also only 2 levels of quality
2) my main point here, relating to the MusOpen.org rant - a quick look on Featured Art reveals that the choice for inclusion isn't strongly based on objective scales. It's clear that eg http://opengameart.org/content/2d-circle-graphic-archive-townbot2bmp is high-quality but isn't included in Featured Art, the same goes for all others of the set - while http://opengameart.org/content/homemade-flame is in Featured Art but isn't high-quality artwork; some half of Featured Art isn't objectively Good - mostly commissions (including mine - one day I'll come back to it and make some better versions of them). Featured Art should be "prestigious", and measured on objective grounds.
Being unempathic & objective may at first sound discouraging to the community, but it can also be encouraging in the way that there is a certain absolute Quality to crave towards, which encourages contributors to improve; and a True Artist or really anyone with a True Specialist mindset is always more interested in the objective value of his own work.
_The problem with entries that should/could be bundled up as one, but aren't_
This is primarily a technical issue, however I'd like to draw attention to just how important it is. The primary way for a browsing user to reach the images he is looking for, is through Search which displays 4*8 images with titles on the screen, representing entries; but see eg http://test2.opengameart.org/search/art?page=2&keys=tileset & http://test2.opengameart.org/search/art?page=3&keys=tileset - I imagine cluttered search pages like that may quickly make a user give up. So, I'd really like to see them grouped up.
Ideally, all images could be formed into groups - and Search would search from among those groups, rather than from among individual images. A game maker with any sensibility of what makes a game look good, will want imagesets and has relatively little use of sole images.
_The problem with art storaged from other projects_
The problem is, there isn't enough of it! Eg: many Wesnoth terrains & portraits, and some weapon icons and units, could be asked permission for and uploaded. The result would be imagesets with higher quality than the commissions have been, as you can only ask for the better images if you want.
_The problem with the quality of commissions_
The commissions have had a lowish quality. Not programmer art, but most of them wouldn't find a use in eg a commercial project - and say what you will, a good game will need to look as if a commercial project - because there aren't enough examples of good quality in uncommercial games to compare to. Maybe doing fewer commissions, and putting more money behind each one, and being pickier with whom to commission from?
_The problem with design bloat_
Images and their titles make up an insignificant portion of the screen currently. Say, 0-10%? I think, for the sections of the site dealing with images such as Search listings and main screen, it should be as much as 70%. Iirc this was to be cured somewhat in OGA 2.0; but I can't find a link to it, so I can't comment much on that.
_Tl;dr_
A version for those of you who scrolled past the text:
I agree completely. I think if OpenGameArt is going to break open, it will be by aggressively pursuing (and showing off) quality.
Not all of us have a good critical eye though. Instead of relying on random users to rank items, should we appoint some users as trusted "critics" to rank entries? At least, some sites separate critics from user scores (e.g. critics rank it X%, users rank it Y%).
DeviantArt shows "most popular" entries for users not logged in, with the option to instead view by most recent. This definitely gives new users a better impression of possible quality. One hard part of that: commercial-quality art comes along rarely, so the OGA frontpage would look unchanged for long spans of time. Harvesting existing projects might help there.
A couple of responses:
First, these are good comments, so I don't want to come off as defensive. That said, many of these things have been considered in the past:
Agreed.
"Most popular" helps, but at least on dA it leads to stuff like pin-ups and popular anime being placed very high (because even if people doesn't think it's objectively all that good, naked skin is still naked skin, right? This leads to clicks, silly as it is...). Daily Deviants are at least in theory better since only a few people are able to submit them (though other users can, and often do, suggest them).
As for grouping of artwork, this has been my major pet peeve more or less since the start of the site, so I hear you.
I think the reason we haven't yet uploaded more from various projects is a combination of being careful (perhaps overly so) of not stepping on anyone's toes and laziness. It's not like you have to be an admin in order to do this though; maybe we should encourage users to do this more?
One other note, on commission quality:
The artists we've commissioned have, as a rule, all been very open to comments. My pixel art skills are iffy at best, so I don't have a perfect eye -- if you have comments on their art that might help them improve, my guess is they'll be happy to hear you out. :)
Bart
I agree on most points too. I'd suggest a +1/-1 (and implicit 0) voting scheme (à la youtube).
In the case of really good art, a huge amount of users would vote +1, everything would be fine. In case of so-so art, you'd probably have most people not voting (thus, "voting 0"), maybe a few say +1 but some others say -1 too. And finally in the case of bad art, lots of people would still just not vote, but hopefully there would still be a few -1.
As for the case of voting +1 because of the author, not the art, if the art really is not good, it should get enough -1 to compensate the +1 from friends. If someone gets half the community to vote +1 on his shitty art, he deserves it :)
This is just my suggestion, as I think having dedicated/trusted persons giving their vote not only adds a lot of administration cost, but also just moves the problem to somewhere else, less visible: who to choose? can you say "no" if someone motivated wants it? etc.
Also, I would rather see that time invested into building a daily or weekly "featured art" section than on grading every single entry, which would become very burdensome if we get more entries.
Also, what I think is seriously lacking, is an "advanced search", I cannot even say what kind of art I want to search! (Or am I being blind?)
Also, what I think is seriously lacking, is an "advanced search", I cannot even say what kind of art I want to search! (Or am I being blind?)
No, you're not being blind. That's a serious oversight on my part that needs to be rectified.
Bart
"The artists we've commissioned have, as a rule, all been very open to comments. My pixel art skills are iffy at best, so I don't have a perfect eye -- if you have comments on their art that might help them improve, my guess is they'll be happy to hear you out. :)"
I definitely agree on this one. Though I have a fair amount of general pixel art experience, I've still got a lot to learn about certain important aspects of the art (animation being one of the major examples). I would definitley appreciate some more specific comments so I can improve the quality of my work.
Hello again, thanks for the responses; I thought on the subject some more, and what I think would be a good "rating" system would be as follows,
3 categories of images: Unpolished, Polished and Featured. (names are temporary)
The 3 categories are ordered, so it's a bit like a 3-star rating system, except that the meaning of each star is well-defined rather than some mystical "1/3 of some quality"; and the rating is not very volatile, the categories should be presented on the site as distinct groups of images; and there would be some rules surrounding them. "Rating art" would be primarily done by the submitters themselves, and then in a small part by the moderators and other people; that should make up for the "Right now, there just aren't very many people rating art" issue.
Each image will belong to one of the 3 categories. When a user submits an image, he/she can choose to have it as polished or unpolished, depending on how good he thinks himself his work is. Eg I would've rated my Weekly Challenge images as unpolished. Also I'd put all my commissions except the Osare weapon icons (that makes, the 2 tilesets + the contest entry) to unpolished as well; I'm honestly not proud of how they turned out, and wouldn't showcase them. I think a good test question is, "Would I show this work as an example of my skills to a potential employer?" A user should be able to change the work between the 2 categories later on as well. Moderators should be able to change the work between all 3 categories, that is, only they pick the Featured ones, as now. To uphold the value of "polished", entries that seem visibly "unpolished" to the moderators, should be moved to "unpolished"; I hope this won't be taken personally. If the "Unpolished" category is large enough - say, more than half of all the images on the site - it's less likely to be taken personally.
* Unpolished: "Would I show this work as an example of my skills to a potential employer? No." pieces, or pieces of a quality level similar to other pieces on OGA whose authors thought "Would I show this work as an example of my skills to a potential employer? No.", or Works In Progress. A lot of Weekly Challenge entries might go here, among others. The main function of the category would be to build community, rather than to provide commercial-quality artwork to a potential viewer. Should be the most numerous category.
* Polished: "Would I show this work as an example of my skills to a potential employer? Yes." pieces, or pieces of a quality level similar to other pieces on OGA whose authors thought "Would I show this work as an example of my skills to a potential employer? Yes."
* Featured: The best images on the site. Should include (legal) ripped artwork from professional games! Right now, only contributions original to OGA seem to be Featured, which makes it less based on true objective value, and much lowers the meaning of Featured.
On Search, by default, answers should come in the order Featured -> Polished -> Unpolished. On the front page, the 3 categories could be presented as separate groups. Maybe, on the front page, Unpolished images shouldn't be shown to people who haven't logged in? And maybe 2x more Featured Images than Polished images, so as to showcase OGA as a place of Quality Stuff to a potential visitor.
As ceninan said on IRC though, lack of bundling interferes with Featuring. Featuring all the pieces from a game separately might be tedious. Also when the entirety of all the images from the game is worthy of Feature, a single spritesheet from that set might not be - so Featuring all the spritesheet entries from a set may give the impression that OGA thinks that some single spritesheet is worthier of Feature than some other images on OGA that are individually better but not Featured - while that doesn't have to be the case.
* * *
@bart,
Thank you for the response; I think I addressed most of your post above;
* "As for art from other projects, I completely agree with you, but uploading all of that art takes time, which I don't exactly have in spades. I have, for the record, asked for volunteers on multiple occasions to upload large amounts of art from projects, but while everyone seems to think that someone ought to do it, nobody really seems to want to do it."
As I said on IRC, a large part of ethically ripping from free games is asking for permission even if the license doesn't demand you to - and a person who cannot claim to speak for OGA, such as myself, will have less of an authority when asking for that permission. Also, lack of bundling makes the task a lot messier/harder.
* "Design bloat: What page are you referring to? The main page, or individual art pages? The main page is always going to have more on it than just art. I am not willing to negotiate on that point. As for individual art pages, they could stand to be redesigned. The latest mockup of OGA 2 (from a while ago) is here: http://oga2.opengameart.org/mockup2/ (note that at the moment I have no design for the individual art pages)"
Of the main page, the mockup does seem better than now (~ 300 000 pixels out of ~ 1 000 000 = 30% in my browser window are under images & their names); but still more space could be saved without stealing any space from the other content (~ 500 000 pixels out of ~ 1 000 000 = 50% in my browser window), here is a slightly changed mockup:
http://i.imgur.com/jTy9w.png
http://i.imgur.com/n4YdL.png
I'll try to post something on the entry preview pages in near future.
Blarumyrran, I like the unpolished/polished/featured idea. Helps gather up the great art but doesn't discourage people from submitting.