Homebrew Optimization Contest -- Brainstorm thread

[bhu]

So I came up with the idea (thanks to Josh's poll) of running homebrew optimization contests.

Here's the rationale:

Here's another idea: we have a huge amount of homebrew on this site.  Why don't we run homebrew-optimizing contests?

1) Few people will be very familiar with the material (the creator should of course be a judge and not a contestant), but the system mechanics WILL be familiar.
2) There are few resources out there (in terms of handbooks and whatnot), so optimizers will be using their skills instead of parroting other peoples' work.
3) It will expose the contestants and kibitzers to awesome homebrew they might not have seen otherwise.
4) Nothing tests a piece of homebrew like a bunch of optimizers trying to break it... so the homebrew designers will get a ton of feedback on their work.

Thoughts?

So, to do this we'll need answers to a few questions:
1) Who's on the organizing committee with me?
2) What forum do we host this in?
3) What are the entry guidelines?  How many levels in a build, or should it be multi-level? What, besides the particular homebrew material being op'ed, is allowed?
4) What are the judging criteria?  How many judges, besides the author? (I don't think the organizing committee should necessarily automatically judge, since I know I'd love to enter a few.)
5) What's the policy for editing the material mid-contest?  Basically, what happens if the contestants discover that a mechanic is broken while they're op-ing?  Does the author get to edit, or do they have to wait until the contest is over?
6) What types of things are being optimized?  Just classes, or feats, etc. as well?  How do we want to handle it for homebrew that involves more than a single thing (like Rituals or Spellshaping)?
7) What are the rewards?

I'll volunteer something from the Ritual system for the first contest.

1: I might be able to help, it depends on how rl events work out for me.
2: This one
3: Set that per individual contest
4: You're assuming the author is an optimizer.  They may not be.  The judges should be people who most definitely know their stuff.
5: Give the author a short period of time to edit, or it goes as is.  If he does edit, extend the contest a few days so everyone can redo their stuff.
6: Vary it up per contest
7: Ummmm....

[veekie]

Actually, you could just quote the material used at the time of contest, as the contest version. That'd be cleaner and less confusing if future edits are carried out(presumably closing exploits) and the contest builds make no sense after.

[Garryl]

Actually, you could just quote the material used at the time of contest, as the contest version. That'd be cleaner and less confusing if future edits are carried out(presumably closing exploits) and the contest builds make no sense after.

This makes excellent sense.

[DavidWL]

So, to do this we'll need answers to a few questions:
1) Who's on the organizing committee with me?
2) What forum do we host this in?
3) What are the entry guidelines?  How many levels in a build, or should it be multi-level? What, besides the particular homebrew material being op'ed, is allowed?
4) What are the judging criteria?  How many judges, besides the author? (I don't think the organizing committee should necessarily automatically judge, since I know I'd love to enter a few.)
5) What's the policy for editing the material mid-contest?  Basically, what happens if the contestants discover that a mechanic is broken while they're op-ing?  Does the author get to edit, or do they have to wait until the contest is over?
6) What types of things are being optimized?  Just classes, or feats, etc. as well?  How do we want to handle it for homebrew that involves more than a single thing (like Rituals or Spellshaping)?
7) What are the rewards?

I'll volunteer something from the Ritual system for the first contest.

1) I'd be willing to help.  Note: I am only an "ok" optimizer.

2) I agree with others who say this should be a separate forum.

3) I think judging criteria should be fixed before entry guidelines.  Unfortunately, some homebrew is targeted towards lower levels, where other homebrew (mystic theurge fix) is usually targeted towards higher levels.  I'd tend to think benchmarks of a build at 7th and 14th level are reasonable.

4) I think:  Flavor, Exposing Broken-ness, full use of the homebrew being used, & optimization.  Also, I don't think the author should be a judge ... make it more impartial.

5) Mid-contest edits --> I'd say that anyone who finds some broken-ness or lack of clarity should get a small bonus when being judged, but that the material should change mid-contest.  So that people share their discovery's as they go, and the author is free to update if desired.

6) I'd argue that what is being optimized is _very_ important.  If it is too small (optimize 1 feat) then the competition doesn't have much meaning, and if it is too big (optimize Frank & K in their entirety) then it may take too much time for people to read and become familiar.  I'd argue that this should be 1 class + a few supporting details (feats, custom skills, whatever).  And if it is more, than this, it not be in scope for the 1st competition.

7) I think this is partly dependent on what mod's can do.  At the very least, bragging rights! 

I think item 6 is very important ... finding the right scope, as well as homebrew that is in some sense comparable will be a tough job.  I'd almost say that the judging committee should approve any submitted homebrew, so that it can all stay very vaguely comparable.

For example, how would you compare a spell point system with a mystic theurge feat?

Best,
David

[sirpercival]

It seems clear that a lot of these questions will need to be answered on a contest-by-contest basis.  I disagree with a few of your answers, though:

3) I think judging criteria should be fixed before entry guidelines.  Unfortunately, some homebrew is targeted towards lower levels, where other homebrew (mystic theurge fix) is usually targeted towards higher levels.  I'd tend to think benchmarks of a build at 7th and 14th level are reasonable.

This will almost certainly change depending on what's being optimized.  How about we have the author suggest breakpoints?  They know the material the best.

4) I think:  Flavor, Exposing Broken-ness, full use of the homebrew being used, & optimization.  Also, I don't think the author should be a judge ... make it more impartial.

I don't think "exposing brokenness" should be a judging criteria.  It's a good thing to have happen, but it falls under optimization.  We'll have to decide whether someone who finds a broken combo should get more points than someone who just optimizes well.  What's the goal?  Also, do we really want "flavor"?  Won't people be less likely to participate if they need to flesh out backgrounds and personalities?

And, the author should DEFINITELY judge.  Why would the author make things less impartial?

5) Mid-contest edits --> I'd say that anyone who finds some broken-ness or lack of clarity should get a small bonus when being judged, but that the material should change mid-contest.  So that people share their discovery's as they go, and the author is free to update if desired.

I'm kind of leaning towards no edits on this one... but the organizers should make a quick check for obvious brokenness before approving a contest.  If the mechanics are so borked that everyone will find a broken combo then it's not a good contest and we should suggest that the author edit.

6) I'd argue that what is being optimized is _very_ important.  If it is too small (optimize 1 feat) then the competition doesn't have much meaning, and if it is too big (optimize Frank & K in their entirety) then it may take too much time for people to read and become familiar.  I'd argue that this should be 1 class + a few supporting details (feats, custom skills, whatever).  And if it is more, than this, it not be in scope for the 1st competition.

I think we'll probably need to set bars for the minimum and maximum amount of content that can be submitted for a contest.  One feat is just not enough... but the entirety of a giant subsystem is way too much.  We'll have to do this on a category-by-category basis.

think item 6 is very important ... finding the right scope, as well as homebrew that is in some sense comparable will be a tough job.  I'd almost say that the judging committee should approve any submitted homebrew, so that it can all stay very vaguely comparable.

For example, how would you compare a spell point system with a mystic theurge feat?

The committee should absolutely approve things.  I thought that was a given.  But why would you compare those two example things?  We can run longer vs shorter contests, for example.

[DavidWL]

This will almost certainly change depending on what's being optimized.  How about we have the author suggest breakpoints?  They know the material the best.

I was under the impression that multiple different pieces of homebrew would be being used in the same competition.  (Example:   Use at least one of these 4 classes and 4 feats).

So that there would be no "one-size-fits-all" answer to this question.  Was that not what you were intending?

Having it be only 1 piece of homebrew makes judging much easier, but it also exposes people to less homebrew and is less varied.

I don't think "exposing brokenness" should be a judging criteria.  It's a good thing to have happen, but it falls under optimization.  We'll have to decide whether someone who finds a broken combo should get more points than someone who just optimizes well.  What's the goal?  Also, do we really want "flavor"?  Won't people be less likely to participate if they need to flesh out backgrounds and personalities?

Hmm.  Frankly, there are some builds which are just "elegant" ...

For example, this is a cool theme build, which isn't uber powerful, but takes a cool and normally poor tactic and makes it good in an "elegant" way:
http://community.wizards.com/go/thread/view/75882/19872622/Challenge:_Optimization_Circus!?post_id=338464874#338464874

How do we judge this?  I'd argue it has both flavor and good optimization, but certainly it is strictly worse than archivist 20 in almost all ways ... and not really even better than warblade 20 ... ?

I think we just have to ask "what is your goal" and "how successfully did you make the build effective for those goals".  With bonus points for innovation.

And, the author should DEFINITELY judge.  Why would the author make things less impartial?

Hmm.  In retrospect, I agree.  My bad.

That said, it may not always be the case that the author is both available and interested in judging.

I'm kind of leaning towards no edits on this one... but the organizers should make a quick check for obvious brokenness before approving a contest.  If the mechanics are so borked that everyone will find a broken combo then it's not a good contest and we should suggest that the author edit.

I'm fine with this.  There can be an open "suggest edits" stage before material is fair game.

The committee should absolutely approve things.  I thought that was a given.  But why would you compare those two example things?  We can run longer vs shorter contests, for example.

I think this is about my thinking that multiple homebrew would be in the same contest, whereas I suspect you were thinking only "one" piece would be included.  While I would prefer multiple "manageable" homebrew, I'm ok with either system.

Best,
David

[veekie]

Hmm, depends on category of contest then. Modular things like martial disciplines, feats and equipment can be part of a mixed contest, but classes(especially base classes) would be better served individually. 'Single' contests are easier to judge though.

Hmm, how about we make a trial run with a single piece of homebrew then? We've plenty of stuff to pick from here.

[DavidWL]

Hmm, depends on category of contest then. Modular things like martial disciplines, feats and equipment can be part of a mixed contest, but classes(especially base classes) would be better served individually. 'Single' contests are easier to judge though.

Hmm, how about we make a trial run with a single piece of homebrew then? We've plenty of stuff to pick from here.

I'm game. 

How about starting a thread for people to nominate base classes.  After the nomination, have a poll to see which base classes people want to work with, and the contest be run with 1 (or more) of the top base classes?  I can probably find a few base classes I'd nominate, etc. 

[bhu]

I know what I'm nominating

[littha]

Thread is up, nominate some classes:
http://www.minmaxboards.com/index.php?topic=2186.new#new

[sirpercival]

So... who officially is in the org committee?  Dredging through the thread, I have the following volunteers (stop me if I'm wrong):

sirpercival
littha
Garryl
Childe
bhu
DavidWL

and the following people who contributed to the discussion but did not explicitly volunteer:

veekie
Noliar
phaedrusxy


Does that sound right?

[veekie]

My availability is sporadic recently(though I'm always sorta around, actually being able to help may be an issue), so best to leave me off the list.

[Bozwevial]

If for some bizarre reason you need more than six to eight people involved in this thing, I'll help out.

I would have stuck my nose in here earlier, but exams.

[bhu]

what exactly does the committee entail?

[DavidWL]

what exactly does the committee entail?

Good question.

I'll either help or submit a build ... let me know what you need more, lol.

In terms of tasks:
* Giving homebrew a once-over (perhaps this should just be done in a thread)
* A small group of people reviewing builds for legality
* rating builds, and perhaps a combined rating for ranking

Is there anything else?

Also, can the mods give cool icons or other perks?

Finally, perhaps we should give the nomination thread 1 or 2 more days, then put up the poll.

Best,
David

[DonQuixote]

On the issue of author judgment, I think a better system would be to have the author as a default judge, but allow him to pass off the role to a proxy.  Writing homebrew doesn't require optimization experience, and I'm pretty sure that there are authors out there who would love to watch optimizers go after their stuff...but might not feel qualified to judge the final results.

[sirpercival]

what exactly does the committee entail?

Good question.

I'll either help or submit a build ... let me know what you need more, lol.

In terms of tasks:
* Giving homebrew a once-over (perhaps this should just be done in a thread)
* A small group of people reviewing builds for legality
* rating builds, and perhaps a combined rating for ranking

Is there anything else?

Also, can the mods give cool icons or other perks?

Finally, perhaps we should give the nomination thread 1 or 2 more days, then put up the poll.

Best,
David

Poll is already up.  We'll have to find out if mods are willing to give out things like that as prizes -- Veekie?

The other thing the organizers will be responsible for is judging, either by doing it themselves or by finding others to do it.

On the issue of author judgment, I think a better system would be to have the author as a default judge, but allow him to pass off the role to a proxy.  Writing homebrew doesn't require optimization experience, and I'm pretty sure that there are authors out there who would love to watch optimizers go after their stuff...but might not feel qualified to judge the final results.

Of course a 'brewer could turn down the chance to judge.  My personal thought is that the author should have some input on who wins, in terms of maybe not rewarding a build that was highly "optimized" by abusing or twisting the rules of the 'brew.

[veekie]

Ok, I checked, what we can be sure to be able to give out is custom titles, though the specifics would still have to be worked out. Other rewards would need more tinkering and checking.

[bhu]

I can do custom homebrew but im not sure what else I'd be capable of right now/

[sirpercival]

OK, were' back here.  We now have a first contest: Oslecamo's Plain Scholar.

Zeroth, we should probably move this to the forum...

First, what are the entry guidelines?  Second, what are the judging criteria?

For the former, I'll reiterate:

{snip} each contest will have two threads, one to describe the contest and one for discussion with the author (asking rules clarifications, giving feedback, etc.).

I agree that the exact rules for entries will depend on what it is that we're testing (class vs feat vs etc.). We'll have to think about what sort of contest is appropriate for each of veekie's categories.  For example, how would people optimize a combat system replacement?  Would "builds" work?  Or would we need to run an arena-style game?

I suggest we start with something rather straightforward, like a D&D 3.5 homebrew class.  Proposed rules:

• Allowed homebrew content -- The class in question, as well as supporting material by the author; all of this will be clearly delineated in the contest thread.  For example, if we ran a contest for one of my Ritual classes, the entries could utilize the rituals, feats, races, and equipment for the Ritual system, but not other homebrew that Garryl, I, or anyone else wrote.  Additionally, nothing new that's written while the contest is running is allowed.
• Allowed other content -- Anything published by WotC.  No Dragon mag, or non-WotC "official" content like spelljammer or athas.
• Build guidelines -- 32 point buy, 1 trait, 1 flaw.  No LA buyoff.  Must use at least 7 levels of the class (PrC or Base), more points for more levels.  Retraining & rebuilding allowed, but no DCFS.
• Entry guidelines -- Present a 20-level build, including explanations of viability at various levels, indicating retraining where appropriate.  The more viable levels for the build (especially early levels), the more points.  Each entry must also include an explanation/justification of build choices (to give the author an idea of why things work together or don't).

We also suggested:

I'd say use the 1xp to 5gp conversion so xp spent in the past that is providing a current benefit counts against WBL. So self crafted magic items would be 50% market price + 20% market price for the xp. Xp from an artificer's own craft reserves don't count against WBL.

I think the MiC rules for combining items to conserve slots should be allowed, as well as custom staffs and things like that.