Comments in this post steered us towards talking about underestimations, specifically underestimations of shows aimed at women. I started out replying in a comment but this is interesting enough for its own post.

Key:
• Series used: this list of 30 series from 2012-2014
• Either intentionally targeted at women or women are assumed to be the majority disc-buying audience (an unavoidably subjective category, yes!)
• Numbers given as all releases combined first, then just BD, then just DVD
• Negative numbers mean underestimations, positive are overestimations
• Vol. 1 estimates only

Female-targeted shows

Format Average Median Max over Max under % releases underestimated
DVD+BD -47.0% -30.9% 34.0% -278.5% 81.5%
BD -57.3% -47.4% 12.4% -278.5% 88.5%
DVD -37.4% -23.6% 34.0% -194.3% 75.0%

Key:
• Everything else, 261 total series
• Details otherwise same; a couple titles that got really wacko due Aniplex pulling titles from Amazon for most of the preorder period are not included

All other shows

Format Average Median Max over Max under % releases underestimated
DVD+BD -4.6% -2.4% 87.3% -145.4% 52.5%
BD -7.4% -4.4% 87.3% -145.4% 57.3%
DVD 0.6% 6.0% 61.4% -142.2% 43.4%

Comments

1. Wow.

2. Okay so potential data issues out of the way: my sample of shows aimed at women is far from perfect. Determining demographics are not always easy, and one of the ways we identify “shows mostly women are buying” is to look at what’s doing well at Animate. So there’s a degree of self-fulfilling prophecy in these categories, skewing the “shows for women” group towards underestimated shows. And I’m sure you’re all itching to argue over this or that categorization choice, but I’d rather not. I think the list is largely okay. All that aside though, the difference is so incredibly stark that I don’t think it can be explained purely by selection bias. As long as I’m not leaving a bunch of shows for women in group 2 – and I don’t think I am! – it should be a usable dataset.

3. Shows in group 1 are underestimated 47% on average, almost exactly 10 times the rate of all other shows. And this holds for median rank as well, with more than 10 times the underestimation at the median. It’s not just outliers at work here.

4. Shows in group 1 are also underestimated in raw numbers much more often – more than 8 in 10, compared to close to half for everything else.

5. There are no extremely overestimated shows in group 1. Only 3 discs are overestimated more than 20% and none were big sellers.

6. BD is underestimated more often than DVD. In fact, all-others DVD is the only category with majority overestimation.

7. To address Musouka’s question about high sellers vs low sellers, it seems like the underestimations in group 1 are not tied to overall sales. Just looking at releases sub-3000 (per format), the average is -45.4% and median is -26.5%, quite comparable to the over-3000 discs (-48.4% average and -35.5% median). The lean towards higher sellers being more underestimated is real, but not particularly strong.

8. Between those group 2 medians being impressively close to 0 and the pretty tight half and half under/over by number of releases, Stalker’s formula sure looks pretty solid, at least in the aggregate. Just not for shows for women!

9 Responses to “Prediction Talk: Stalker underestimations, female-targeted shows vs the rest”

  1. Avatar primadog says:

    A “Wow! signal” for anime sales wonks if we ever see one. Any possibility of sharing the STALKER vs week 1 Oricon data of the 30 Fujoshi shows to the public?

  2. Avatar musouka says:

    Wow, this is really fascinating data. (I also can’t think of any problems with your list at a glance.) It also really goes to show you that when in doubt, you should run the numbers.

    It’s kind of funny, because in a way I’m surprised, but in another way, I’m really not. We’ve been seeing this pattern more and more lately, no matter how much people want to deny it. Stalker just isn’t a good reflection of the modern female anime fans’ buying patterns at this point in time…

    • Avatar primadog says:

      And given how big the variance the underestimate is, I don’t foresee how a simple fix to STALKER can ever hope to improve its accuracy for this demographic.

      • Avatar musouka says:

        Yeah, I was just thinking that. I can’t imagine how Stalker could even begin to account for it, so it’s not like I blame the site in particular. It more means that people should stop taking Stalker as gospel–well, they should do that anyway regardless of demographic in question–in particular for series aimed at women.

        • something something says:

          I’d love to see Stalker follow the Animate 200 page for a long time and, like we were discussing in some other threads, see what they could make of a big enough dataset. Conceptually it would be simple – compare how the normal estimate performed vs how a series ranked on average in the 200, and see if there’s any sort of correlation once you’ve got a bunch of examples.

          I mean we know there’s a correlation, we just need more data to get a better sense of how predictable a correlation it is. If nothing else it’s the lowest hanging fruit, unlike trying to figure out what the hell Gamers’ rankings mean. Even HMV with their large selection seems to have very little predictive power. I feel like the last time they were useful at identifying underestimated titles was Chuunibyou s1! In some ways, adjusting for the Animate audience may be the easiest goal (in a list of not necessarily easy goals, granted), because it is a bigger and more consistent factor.

          But anyway the new comparison page http://rankstker.net/index_news_comp.cgi just has so much potential for this sort of thing, but only if it can be analyzed in an intelligent way (which only Stalker is in a position to do with all its harvesting) rather than us just going by our high level impressions.

          • Avatar musouka says:

            I think part of the issue is, “If it ain’t broke, don’t fix it.” Yes, I’d love for there to me more accurate data in regards to female-oriented shows, but as these numbers show, Stalker the way it is now is pretty good at ballparking for most series the way it is…

            On the other hand, if this boom with female viewers keeps up–and I haven’t seen anything that makes me think it won’t–then these numbers become more and more important in general. I think you’re right when you say that Animate will probably be the key there, but I’ll leave Stalker to figure out exactly how much weight it should be given.

            • something something says:

              I think it’s becoming more important because it also goes beyond shows with an Animate bias. This ongoing silliness with Aniplex releases on Amazon is looking to cause a very similar effect, and love it or hate it Aniplex still puts out a lot of the best-selling titles. If SAO II, Aldnoah, and P4GA all replicate Mahouka’s performance then we’ve got to expect the same for Fate/Stay, Durarara!! 2, Imas Cinderella Girls, and so on.

              It’s not “broken” by any means, but there’s still a lot of room for adjustment and improvement. They can explore additional, alternative options without changing the existing methods, until such time as the new methods mature and prove themselves better than what we’ve got now.

              Might as well try!

Leave a Reply