LESSWRONG
LW

Comment Permalink
bglass3y1

I made an account for your challenge.

I plotted items by color and looked at minimums, maximums, and averages. Yellow items consistently provide just under twenty mana while blue items always provide about the same as the reading, except that they sometimes provide twenty extra. I was too lazy to try to figure out the red or green items. Given what I know, I can submit the blue items HoC, RoJ, and PoH to get about 140 mana for 101 gold, and call it a day.

However, I also noticed that no blue item provided over 60 mana. I will add in the yellow items PoP and WoJ for a margin of safety. Painfully, that brings the total to 177. Combined with the chance that other items will provide extra it should be enough for some confidence.

The Wizard Wakalix did not inform me of the reliability of blue items; they merely called their device a liar. (Perhaps they aren't yet aware of the pattern because they don't utilize the arcane magic known as python). I may as well include a letter that outlines my findings regarding yellow and blue items. That way, the next time Wakalix goes to the caravans they can make use of the information, and the next time they want to hire someone I will be their first thought.

If I were devious, I might try to include red and green items to obfuscate the reliability of blue for predicting mana cost. Sending three blue items at once will surely let the cat out of the bag. However, If I expected deviousness, this setup is an excellent way to deceive an errand-runner into providing free magic items. I'll count on the norms regarding my work to prevent either of those outcomes. I may check with other errand runners to make sure Wakalix isn't running a scam, just in case.

Edit: Reading the comments allowed me to notice I mixed up the direction of the error: blue items read 20 over, not provide 20 over. Well. Good thing I included a margin.

See in context
D&D.Sci
by abstractapplic
2 min read12th Jan 202128 comments

51

Exercises / Problem-SetsD&D.SciWorld Modeling
Frontpage

The day’s task shows up in an envelope, and not in glowing purple letters emblazoned across the inside of your eyelids, which is usually a good sign. The owl that brought it looks on with equanimity as you read its master's message:

Hello,

I hearde that you do odde jobs for Wizards. I neede 120 mana for a ritual but cannot leave my Tower righte now. Go to the caravans in towne and buy enough magic items that I can gette that much by sacrificinge them.

My Owle has a pouch. It is biggere inside than oute. Putte the things in it ande she will carrye them back.

Enclosed is my Thormo Tharmeu Magic Sensing Device. It usually lies but is probably bettere than guessinge. Returne it when you are done. Enclosed is also a list of 836 itemse I sacrificed and what coloure they glowed and how muche mana I gotte and what the Thau Lying Box said when I pointede it at them. I like lists.

The pouch contains 200 gold pieces. You may keepe what coins are lefte over. If I do notte gette at leaste 120 mana from the things you sende me, you shalle owe me 200 gold pieces.

Goodbye,

Wakalix the Wizard

PS: If you do not accepte the jobbe, I bid you sende the Owle and the gold back before sundown, that I may finde another to charge with it.

Your spirits lift with every line. Clear objectives, payment in advance, acknowledgement that you have the right to refuse the task, no threats of involuntary transformation, no random tangents about world domination or beard care, handwriting legible, capitalization not entirely random . . . this is one of the good clients. And if you make clever enough use of the list he provided, you suspect you could end up taking home a decent fraction of that 200gp once this day’s work is done. With a song in your heart, you depart for the travelling caravans and their magic items.

The selection of artefacts that greets you is as follows:

Item nameGlow colorThaumometer readingPrice
Longsword of Wounding +2Red1466gp
Warhammer of Justice +1Yellow541gp
Hammer of CapabilityBlue3535gp
Pendant of TruthRed4038gp
Ring of Joy +5Blue2932gp
Warhammer of Flame +2Yellow4865gp
Battleaxe of GloryBlue723gp
Plough of PlentyYellow1235gp
Saw of Capability +1Green1635gp
Amulet of Wounding +2Green5035gp
Pendant of HopeBlue7734gp
Pendant of Joy +4Green4239gp

 

Will you accept Wakalix’s errand? If so, what will you buy?

I’ll be posting an interactive letting you test your decision, along with an explanation of how I generated the dataset, sometime this Sunday. I’m giving you a week, but the task shouldn’t take more than a few hours; use Excel, R, Python, tarot readings, or whatever other tools you think are appropriate. Let me know in the comments if you have any questions about the scenario.

If you want to investigate this collaboratively and/or call your decisions in advance, feel free to do so in the comments; however, please use spoiler tags when sharing inferences/strategies/decisions, so people intending to fly solo can look for clarifications without being spoiled.

Exercises / Problem-SetsD&D.SciWorld Modeling
Frontpage

51

Previous:
D&D.Sci Evaluation and Ruleset
10 comments41 karma
Next:
D&D.Sci II Evaluation and Ruleset
6 comments33 karma
Mentioned in
New Comment
29 comments, sorted by
top scoring
Click to highlight new comments since: Today at 8:19 AM
[-]GuySrinivasan3y13

tl;dr:

Blue Pendant of Hope, Blue Hammer of Capability, Yellow Plough of Plenty, Yellow Warhammer of Justice +1, gaining 124+ mana for 145gp.

Do not gamble with green items or the red pendant of truth; the value of a good, lucrative client with a somewhat useful sensing device and your own credibility is worth much more than some gold pieces today.

okay, lemme read:

Regularities found:

  • blue items register +/- 1 on the thaugister, except jewelry gets a +22
  • weapon mana is at least 10x the bonus and no more than 10(x+1) the bonus
  • (axes and hammers are tools, not weapons)
  • yellow items yield 18-21 mana
  • green items are capped at 40 mana and their thauyard readings seem very uncorrelated
  • actually all jewelry gets a +22 to its thaudout
  • jewelry-adjusted blue-adjusted thaulues are at most 60
  • red mana is always some product of 2s, 3s, and 5s
  • green items always yield an even amount of mana
  • non-blue items have thauras whose only prime factors are 2, 3, 5, or 7
  • green items yield mana whose prime factorization's non-2 exponents sum to at most 2, and only if exactly 1 from bases greater than 5
  • some identical red and green items have wildly different mana/thauata
NameGlowThauxyPriceMana
Longsword of Wounding +2Red1466gp

20, 24

Warhammer of Justice +1Yellow541gp

18-21

Hammer of CapabilityBlue3535gp

34, 36

Pendant of TruthRed4038gp

2^a x 3^b x 5^c

Ring of Joy +5Blue2932gp

6, 8

Warhammer of Flame +2Yellow4865gp

18-21

Battleaxe of GloryBlue723gp

6, 8

Plough of PlentyYellow1235gp

18-21

Saw of Capability +1Green1635gp

2 x A

Amulet of Wounding +2Green5035gp

2 x B

Pendant of HopeBlue7734gp

54, 56

Pendant of Joy +4Green4239gp

2 x C

200gp, 120m? We can get that with Blue Pendant of Hope, Blue Hammer of Capability, Yellow Plough of Plenty, Yellow Warhammer of Justice +1, gaining 124+ mana for 145gp. Definitely take the job, and try to figure out on the road what's up with the Red Pendant of Truth and all the Green items. A good Green item saves ~30gp, but if we can know the Red Pendant of Truth is great we could save even more.

Crucially, no probabilistic choices should be made, for three reasons:

  • You have no guarantee about the distribution of item stats presented. They are very likely not uniform random. In fact meta-you suspects that there was once a list of 1000 items and 164 got filtered out.
  • You have a good client who is handing you 55gp for a day's work. This is an excellent situation. It is far better to keep this client than to try to eke out additional gold for this particular job.
  • Further, you have valuable information. You know what the client's thaudget does with jewelry and items with a blue glow, and you know how weapon mana and bonuses are related, and you know the yield of items with a yellow glow, and you know the maximum yield of items with a green glow. You can use this information in future deals with the client, or, if you can arrange a more profitable deal for the information itself, perhaps land a major score. But this second possibility is far, far less likely if you accidentally damage your credibility by supplying less than 120 mana.
[-]Measure3y8

Can I use the Pendant of Truth to make the Thaumometer readings more accurate?

[-]abstractapplic3y4

Fun thought, but no. In fact, none of the available magic items can help you in that way.

[-]gjm3y7

Proposed buy (no explanations but may still be spoilery; there is a lot I still don't understand so I suspect one can do better):

 WH o Ju, Pl o Pl, Ha o Ca, Pe o Ho. I expect a little under 130 mana, for a cost of 144gp.

Explanations (definitely spoilery):

 Yellow-glowing things get 18-21 mana; I haven't found patterns beyond that. Green-glowing things get 2-40 mana, always an even number; I haven't found patterns beyond that. Red-glowing things get 2^a 3^b 5^c mana; other than the fact that somehow we never get >96 even though we separately get 64, 27, 5, I haven't found patterns beyond that. Blue-glowing things get highly variable mana, also favouring small prime factors though 7 occurs; for these (and only these) the thaumometer gives plainly useful information, yielding the true mana gain +-1 except that items you wear yield a number too high by 22. So the two cheaper yellow items are pretty good value, as are the highest-thaumperature blue ones even though one of them is overrated. We should get at least 18+18 for the yellow ones and at least 34+54 for the blue ones.

But

I suspect there may be more going on than I yet understand with the red and green items, for which at present I don't think I know anything useful. And maybe the finer details of the yellow and blue ones are predictable too.

[-]ejacob3y4

I came up with the same answer, though I just excluded the 

colors

that weren't amenable to nice linear models and didn't notice the patterns you did.

[-]gjm3y2

Meta: there's one word in that comment that's kinda spoilery and you should maybe spoilerize it.

[-]ejacob3y1

Good catch, thanks!

[-]Jemist3y3

Red or green weapons i.e. swords, longswords, battleaxes (not axes or hammers though) seem to have a mana scaling dependent on their +n modifier (although green weapons have a drop-off at higher modifiers. It appears to be a clear enough pattern that it's not a statistical artefact. I've not found anything else about the tools or jewellery though.

[-]Pablo Repetto2y6

I've written my independent work into a post. There is also a preface with some explanation as to the choice of tools.

[-]gjm3y5

Wakalix the Wizard. Slogan: "Wakalix Maketh It Goe!"

[-]Ericf3y2

Can you buy more than one of an item?

[-]Ericf3y1

Then I suppose I shall have to pass, as these items are too expensive, even disregarding the downside risk if they come in below average.

[-]gjm3y2

Then you are missing out. I have only a partial understanding of the phenomena so far, but I already have a set of four items that I think should pretty reliably get at least 120 mana for a total price below 150gp.

[-]Ericf3y1

I see I had a transcription error where I thought the blue Pendant had a reading of 34 instead of a gold cost of 34. In revision, I would go ahead and send in 5 items, spending 177 gold, just in case.

[-]Yonge3y1

I came up with this:

// Plotting a graph of reading v colour reveals the following.
//- Blue somewhat erratic but definite trend of increasing mana with increasing reading
//  The erratic part is in the 22 - 63 range, before and after steady increase/decrease.
//- Green  seems to fluctuate in the 2- 40 range regardless or reading
//- red erratic all over the place, no consistent pattern
//- yellow seems to fluctuate in the 18 - 21 range regardless of reading
//
// Green has an average mana of 21, red 25 , though green seems to have fewer with really low values.
//
// There is no obvious correlation based on the items name.
// Eliminating obviously uneconomic items suggest the following are realistic:
//Pendant of Hope         54 mana      34 gold    BLUE
//Ring of Joy             10-30 mana   32 gold    BLUE
//Hammer of Capability    15-35 mana   35 gold    BLUE
//Warhammer of Justice +1 18-21 mana   41 gold    YELLOW
//Plough of Plenty        18-21 mana   35 gold    YELLOW
//Saw of Capability +1    avg 21 mana  35 gold    GREEN
//Amulet of Wounding +2   avg 21 mana  35 gold    GREEN
//Pendant of Truth        avg 25 mana  38 gold    RED
//
// Pendant of Hope is obviously the best. Could reach target with near 100 percent certainty with top 5 items
// leaving me with 23 gold.
//
// Pendant of Hope + Saw of Capabilty + amulet of Wounding + Pendant of Truth would leave me with 58 gold
// if it worked, but would probably fail just under 50 percent of the time. This could be reduce significantly
// by paying 32 gold, but that would only leave me with 3 gold more, and still looks less certain.
//
// As avoiding being in debt by 200 gold is probably much more important to me than gaining 25 gold I will go
// for what looks to be the safe option of:
// Pendant of Hope
// Ring of Joy
// Hammer of Capability
// Warhammer of Justice + 1
// Plough of Plenty
// And hopefully 23 gold.
 

[-]bglass3y1

I made an account for your challenge.

I plotted items by color and looked at minimums, maximums, and averages. Yellow items consistently provide just under twenty mana while blue items always provide about the same as the reading, except that they sometimes provide twenty extra. I was too lazy to try to figure out the red or green items. Given what I know, I can submit the blue items HoC, RoJ, and PoH to get about 140 mana for 101 gold, and call it a day.

However, I also noticed that no blue item provided over 60 mana. I will add in the yellow items PoP and WoJ for a margin of safety. Painfully, that brings the total to 177. Combined with the chance that other items will provide extra it should be enough for some confidence.

The Wizard Wakalix did not inform me of the reliability of blue items; they merely called their device a liar. (Perhaps they aren't yet aware of the pattern because they don't utilize the arcane magic known as python). I may as well include a letter that outlines my findings regarding yellow and blue items. That way, the next time Wakalix goes to the caravans they can make use of the information, and the next time they want to hire someone I will be their first thought.

If I were devious, I might try to include red and green items to obfuscate the reliability of blue for predicting mana cost. Sending three blue items at once will surely let the cat out of the bag. However, If I expected deviousness, this setup is an excellent way to deceive an errand-runner into providing free magic items. I'll count on the norms regarding my work to prevent either of those outcomes. I may check with other errand runners to make sure Wakalix isn't running a scam, just in case.

Edit: Reading the comments allowed me to notice I mixed up the direction of the error: blue items read 20 over, not provide 20 over. Well. Good thing I included a margin.

[-]Measure3y1

An average item yields about 20 mana, so six average items would be needed to yield 120 mana. An average item costs about 40gp, so six average items costs about 240gp. Since I only have 200gp I would need to be pretty confident of my choices to expect to make a profit from this. The expected profit for any combination of items is 200*P(success) - cost, so for a purchase to be profitable, it is necessary that P(success) > cost/200.

I estimated the optimism of the Thaumometer for each item by averaging the optimism scores of each trait (average of ln of reading/actual for items with that trait). I used this optimism estimate to calculate a corrected reading for each item. I also calculated a separate mana score for each item by averaging the mana scores of each trait. Finally, I averaged the corrected Thaumometer score with the mana score for each item to get a final guess for its mana yield.

I sorted the list of available items by efficiency (final guess divided by price). [10, 11, 3, 12, 4, 7, 6, 5, 9, 8, 1, 2]

The profit-maximizing strategy should be some number of the most efficient items with at least 120 total expected mana that cost no more than 200gp. As it turns out, there is only one such number of items (5). This combination yields an expected 128 mana and costs 181gp for a profit of 19gp. However, for any purchase to be profitable, the probability of success has to be greater than (purchase price)/(budget). In this case that is 181/200 or 90.5%. Given that the expected mana yield is only barely above the threshold, I think I'm only about 65% confident of success - nowhere near 90%.

My choice, therefore, is to decline the offer and return the 200gp.

[-]Measure3y1

I feel silly now after looking at gjm's answer. I had actually sorted the list by type, attribute and modifier already without finding obvious patterns, and I had planned to do color as well, but then I took a break and when I came back I forgot to look at the colors.

[-]philip_b3y1

Do we know that the old and the new records are i.i.d.?

[-]abstractapplic3y2

From your knowledge of wizardly adherence to habit, the way he never suggested getting items anywhere but the caravans, your knowledge of local markets, and the fact that everyone selling seems to recognize the Owle owl following you, you can be pretty sure that Wakalix bought all 836 of the items on his list from the same source(s) you're currently considering.

(Regarding the other implications of your question, I neither confirm nor deny anything.)

[-]abstractapplic3y1

Somehow forgot to link to dataset in OP, fixed now.

[-]philip_b3y0

Solution:

I send the owl back to Wakalix with all of his 200 gp and the following letter attached

Dear Wakalix the Wizard, Unfortunately, the items that are currently sold by the caravans, can't provide 120 mana for 200 gp. If you are in dire need of 120 mana, please provide me 300 gp instead of 200 gp.

Explanation:

I fit auto-sklearn to the training data, got 7.44 mana mean absolute error and got the following predictions for the test data:

item enchantment refinement color thaum y_pred price y_pred/price
Longsword Wounding 2 red 14 24.8726 66 0.376857
Warhammer Justice 1 yellow 5 18.6423 41 0.454691
Hammer Capability 0 blue 35 12.0596 35 0.344561
Pendant Truth 0 red 40 12.0792 38 0.317874
Ring Joy 5 blue 29 17.9164 32 0.559888
Warhammer Flame 2 yellow 48 20.3161 65 0.312555
Battleaxe Glory 0 blue 7 8.35728 23 0.36336
Plough Plenty 0 yellow 12 19.5171 35 0.557631
Saw Capability 1 green 16 19.4064 35 0.554468
Amulet Wounding 2 green 50 23.8462 35 0.681319
Pendant Hope 0 blue 77 13.0694 34 0.384395
Pendant Joy 4 green 42 29.5016 39 0.756451

From these predictions (the y_pred column), it seems I can't get 120 mana for 200 gp. I could get about 110 mana by buying Amulet of Wounding +2, Pendant of Joy +4, Ring of Joy +5, Plough of Plenty, Saw of Capability +1 for 176 gp. If I had a little bit more gold, I could add Warhammer of Justic +1, that'll be approximately 129 mana for 217 gp, but that would be risky, I could easily end up indebted to Wakalix.

[-]magfrump3y2

Welp, I spent five minutes plus trying to switch to the markdown editor to fix my spoilers and failed. Giving up now.

[-]philip_b3y1

After reading other comments I was surprised. I wondered if auto_sklearn gave such bad estimates because I encoded categorical features myself in a way which is not very suitable for it. I wanted to see if I can tinker with it to get it to give me a better answer. I let it choose how to encode categorical features on its own. I got a better mean absolute error and got predictions, which are very different for Hammer of Capability, Pendant of Truth, Ring of Joy, Pendant of Hope:

item enchantment refinement color thaum y_pred price y_pred/price
Longsword Wounding 2 red 14 27.306131 66 0.413729
Warhammer Justice 1 yellow 5 19.637162 41 0.478955
Hammer Capability 0 blue 35 30.566119 35 0.873318
Pendant Truth 0 red 40 22.123613 38 0.582200
Ring Joy 5 blue 29 6.358711 32 0.198710
Warhammer Flame 2 yellow 48 20.296373 65 0.312252
Battleaxe Glory 0 blue 7 6.949717 23 0.302162
Plough Plenty 0 yellow 12 19.923753 35 0.569250
Saw Capability 1 green 16 18.042059 35 0.515487
Amulet Wounding 2 green 50 22.919911 35 0.654855
Pendant Hope 0 blue 77 54.058931 34 1.589969
Pendant Joy 4 green 42 25.158531 39 0.645091

With these new predictions, I think I can buy Pendant of Hope, Hammer of Capability, Amulet of Wounding, Pendant of Joy, and Pendant of Truth to get approximately 154.83 mana for 181 gold. That's what I do. Although I understand that this is cheating, since I've seen other people's answers.

[-]jaspax3y-4

Though I didn't participate in this exercise, I enjoyed reading about it and looking over the answers below. It put me in mind of a particular meta-point, which is that predictability turned out to be the key. The most profitable answers all hinged on noticing which categories the thaumatometer gave accurate readings for, and using those to minimize uncertainty.

[-]abstractapplic3y3

I'm glad you got what I was going for. However, you should probably spoiler that meta-point, so people trying this challenge in the future can look for clarifications in the comments without being spoiled.

[+][comment deleted]3y1
Deleted by ejacob, 01/13/2021
Reason: messed up spoiler tags
Moderation Log

New to LessWrong?

Getting Started

FAQ

Library