Author Topic: Indexes Project  (Read 9418 times)

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Indexes Project
« on: December 29, 2012, 09:37:03 PM »
Last update: 28th April, 2013

Index Project
What is it?
It's an index that lets you look up the location of any feat, class, spell, monster etc. so you don't have to trawl through all the books
AND it's an index that will list everything in a category, whether that category be 'Aquatic Outsiders', 'Bard Necromancy spells', 'Vestiges that give bonuses to Diplomacy' or 'The entire index of Complete Adventurer'.

Current status?
Pre-Alpha
Lacking all but the most basic documentation, key features missing.
Newest features:
  • Double-click to edit a returned result. Now you can fix errors when you see them.
Developing:
  • To be decided
Current System Requirements
Works only on:
  • Chrome
  • Firefox
  • Internet Explorer with the Google Chrome Frame plugin (auto-linked)
Common issues:
It won't work in "Private Browsing Mode"/"Incognito Mode". It's an offline web application, it needs to be able to store data on your computer.
Where do I get it?
Try it here:
Downloadable (847KB archive) (Recommended)
Online version

Online version warning:
The first load may be slow. After that it should be cached in your browser.
Dropbox's web cache will want to throttle the number of requests your browser makes. It may prove impossible to cache the application from Dropbox.

Only looking for the latest data? 2013-04-28
Download the latest data

Individual data files can be found here. (Likely out of date, yet to be automated.)

How do I contribute?
Okay, let's split this into two parts. I'll do the code and everyone else can collaborate to do the "type everything into a spreadsheet and save it as a comma-separated values file (.csv)" boring data entry part.

Collaborate?
Everything that can be imported into the application can be exported again. You can take an entire snapshot of it (JSON format) to share, OR export the data piece by piece in CSV format.

Importing a snapshot will overwrite anything you already have imported though!

What needs doing?
The current aim is to cover all the D&D3.5 default setting books.
Future goals will probably include things like: D&D3 Eberron+ Faerun, D&D3.0, D&D4E, D&D5th

Data needed:
Spells
Items
Soulmelds
Cityscape (pp. 66-68)
Complete Mage (pp. 95-122)Complete Mage (pp. 125-136)
Dragon Magic (pp. 64-75)Dragon Magic (pp. 92-97)Dragon Magic
Fiendish Codex I (pp. 90-96)
Fiendish Codex II (pp. 100-106)
Magic of Incarnum (pp. 98-106)Magic of Incarnum (pp.108-114)Magic of Incarnum
Tome of MagicTome of Magic
Magic Item Compendium
Planar Handbook (pp. 78-84)
Complete Champion (pp. 131-144)
Tome of Battle
Eberron (all the books)

  • Base Classes (all restrictions)
  • Feats (all prerequisites, types)
  • Magic items (see above)
  • Prestige Classes (all prerequisites)
  • Spells (see above)
  • Tome of Magic: Mysteries, Utterances
What are the system requirements?
I'm aiming for the following browser support:
  • Google Chrome and Chromium
  • Mozilla Firefox
  • Microsoft Internet Explorers with the Google Chrome Frame plugin installed.
  • Opera, once they switch to WebKit.
See current status above for current compatibility.

Where do I report bugs?
At the github project page here
« Last Edit: April 28, 2013, 10:16:29 AM by Psithief »

Offline altpersona

  • Legendary Member
  • ****
  • Posts: 2000
  • #78
    • View Profile
    • You are here
Re: Indexes Project
« Reply #1 on: December 29, 2012, 09:50:51 PM »
dndtools.eu has spells, feats, skills, classes.

surreal has almost everything else. (not really but its the bomb)
The goal of power is power. - 1984
We are not descended from fearful men. - Murrow
The Final Countdown is now stuck in your head.

Anim-manga still sux.

Offline sirpercival

  • Epic Member
  • ****
  • Posts: 10855
  • you can't escape the miles
    • View Profile
Re: Indexes Project
« Reply #2 on: December 29, 2012, 10:03:18 PM »
Are you including homebrew, or just published material?  I know captnq and zook1shoe both have spell databases.
I am the assassin of productivity

(member in good standing of the troll-feeders guild)

It's begun — my things have overgrown the previous sig.

Offline Maat Mons

  • DnD Handbook Writer
  • ****
  • Posts: 1203
  • What is a smile but a grimace of happiness?
    • View Profile
Re: Indexes Project
« Reply #3 on: December 29, 2012, 10:06:04 PM »
ImarvinTPA has several databases.  The spells can be downloaded as a spreadsheet.  Nothing else can be. 

Most of the old Crystal Keep stuff can still be found here

This guy has a partial database. 

If you want the Master +0 LA Race List III as a spreadsheet, I can send it to you. 

Offline kitep

  • DnD Handbook Writer
  • ****
  • Posts: 1948
  • Lookout World!
    • View Profile
Re: Indexes Project
« Reply #4 on: December 30, 2012, 01:18:01 AM »
While you're at it, how about a complete list of magical locations and skill tricks?  You know, because you don't have enough on your plate already   :D

Offline Maat Mons

  • DnD Handbook Writer
  • ****
  • Posts: 1203
  • What is a smile but a grimace of happiness?
    • View Profile
Re: Indexes Project
« Reply #5 on: December 30, 2012, 02:16:16 AM »
Here's a pretty good list of magical locations. 

Here's a list of teamwork benefits.  It doesn't include the ones from The Forge of War. 

Offline nijineko

  • DnD Handbook Writer
  • ****
  • Posts: 2413
  • two strange quarks short of a graviton....
    • View Profile
    • TwinSeraphim
Re: Indexes Project
« Reply #6 on: December 30, 2012, 04:14:39 AM »
the dragondex is pretty useful as well.

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #7 on: December 30, 2012, 09:51:49 PM »
Are you including homebrew, or just published material?
I figure as long as everything can be tied to its source, the user can just select what sources they want to search on.
(Or what category of sources).

It will just be HTML, Javascript, maybe some SVG.
I'm just building the engine to import/export, display, search and sort.
It is an Index engine, though. I'll likely discard parts of sources that contain the entire descriptions.

That won't stop anyone from importing more data than I intend. :)

Offline awaken_D_M_golem

  • Epic Member
  • ****
  • Posts: 7639
  • classique style , invisible tail
    • View Profile
Re: Indexes Project
« Reply #8 on: December 31, 2012, 03:26:16 PM »
« Last Edit: December 31, 2012, 03:28:10 PM by awaken_D_M_golem »
Your codpiece is a mimic.

Offline SorO_Lost

  • Epic Member
  • ****
  • Posts: 7197
  • Banned
    • View Profile
Re: Indexes Project
« Reply #9 on: February 03, 2013, 06:59:47 PM »
*Warning: Finishing this project may provoke propositions of a sexual nature from strangers.*

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #10 on: February 08, 2013, 05:03:34 AM »
Good progress has been made, but I'm short quite a few pieces of data for a comprehensive search. (See first post)

Search rules are as follows:
  • Multiple search terms are intersected together. (AND)
  • Search terms preceeded with a '+' are instead unioned. (OR)
  • Search terms preceeded with a '-' subtract matching results. (AND NOT)
  • There is currently no support for grouping, e.g. (this OR that) AND "that other thing"

  • A data store and a property can be specified as part of the search term. These are separated by colons. (optional)data:(optional)property:searchTerm
  • If subsequent search terms do not reset the data store/property they will apply to subsequent terms.
  • Resetting is achieved by prefixing the term with a colon.
  • Whenever the data store is reset the property is also reset.
  • To search using multiple words as one term, surround the term in double quotes.
Example searches:
Everything*
Level 1 bard spells that aren't mind-affectingspells:"bard 1" -mind-affecting
All eldritch essence invocations for warlocksinv:warlock inv:essence:true
All manoeuvresman:*
Strike manoeuvresman:type:strike
Crusader non-stance manoeuvresman:crus -stance
Large Monstrous humanoidsmon:Large "Monstrous hum"
Creatures with the Good subtype that aren't Outsidersmon:Good -Outsider
Teamwork Benefits that require both Listen and Spot to learnteam:req:listen spot
Spells in the Sun and Healing domains+spells:class:Sun +Healing
All spells and powers with the Creation subschool/subdiscipline:sub:creation
All spells and powers with the Creation subschool/subdiscipline with medium range:sub:creation :range:medium
Everything in the book 'Complete Adventurer':source:cad

Sorting:
  • Sorting does not have partial property matching. 'nam' is not a substitute for 'name' and so on.
  • Sorts properties left to right with quicksort.
  • Sort direction can be inverted for any property be preceeding it with a dash. ('-', not em-dashes or en-dashes).

Here's a pic of the prototype in action:
« Last Edit: April 07, 2013, 08:07:49 AM by Psithief »

Offline bhu

  • Uncle Kittie
  • DnD Handbook Writer
  • ****
  • Posts: 16305
  • Fnord bitches
    • View Profile
Re: Indexes Project
« Reply #11 on: February 08, 2013, 12:25:21 PM »
dndtools.eu has spells, feats, skills, classes.

surreal has almost everything else. (not really but its the bomb)

surreal??

Offline sirpercival

  • Epic Member
  • ****
  • Posts: 10855
  • you can't escape the miles
    • View Profile
Re: Indexes Project
« Reply #12 on: February 08, 2013, 12:29:53 PM »
I am the assassin of productivity

(member in good standing of the troll-feeders guild)

It's begun — my things have overgrown the previous sig.

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #13 on: February 17, 2013, 11:04:41 AM »
Progress update post removed!
« Last Edit: February 25, 2013, 08:17:43 AM by Psithief »

Offline nijineko

  • DnD Handbook Writer
  • ****
  • Posts: 2413
  • two strange quarks short of a graviton....
    • View Profile
    • TwinSeraphim
Re: Indexes Project
« Reply #14 on: February 17, 2013, 11:33:18 PM »
page loads, text can be entered, search triggered, but otherwise does not work on safari, as nothing displays.

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #15 on: February 18, 2013, 03:18:45 AM »
*thumbs up* That's totally expected because Safari is terrible.
http://caniuse.com/#compare=chrome+24,safari+6
Safari does not have IndexedDB.

Should it tell you your browser is not adequate? Of course! but proof-of-concept is way before even alpha, let-alone beta software. :)
The application will let you know if your browser doesn't meet the requirements.
I can put in the shim that lets one use WebSQL in place of IndexedDB but I will have no way of testing it because I do not have Safari 6.
I had a go at using some shim code but it's not quite good enough.
« Last Edit: February 24, 2013, 11:09:57 PM by Psithief »

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #16 on: February 21, 2013, 12:02:37 PM »
Progress:
- magic items from http://chet.kindredcircle.org/ imported.

Problem:
- no magic items from the Magic Item Compendium included.

Solution:
- wait until someone helps out?

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #17 on: February 24, 2013, 08:38:16 AM »
According to the comments in the following blog post, Microsoft is not going to make any attempts to keep IndexedDB up to date. The result is that support for IE10 must be dropped.
Blog post
The Google Chrome Frame plugin will be automatically suggested for any users of 32-bit Internet Explorer.
« Last Edit: February 24, 2013, 10:08:31 AM by Psithief »

Offline nijineko

  • DnD Handbook Writer
  • ****
  • Posts: 2413
  • two strange quarks short of a graviton....
    • View Profile
    • TwinSeraphim
Re: Indexes Project
« Reply #18 on: February 26, 2013, 02:11:37 PM »
i have a master list of almost every psionic power in 3.x d&d official sources. i'm short a few as i got lazy and haven't finished. over 700 main entries.

Offline Psithief

  • Lurker
  • *
  • Posts: 35
  • Can I haz your PP?
    • View Profile
Re: Indexes Project
« Reply #19 on: February 26, 2013, 02:36:49 PM »
Wasn't your list what I used?
It didn't have any page numbers, unfortunately. It was also missing class entries for all the Complete Psionic lists (lurk and mantles).
I put those all in by hand, added psychic rogue's classlist to the powers too.