State of the Sanqui Randomizer

The BS that we all know and love
Post Reply
Sanqui
Jr. Trainer
Posts: 90
Joined: Thu Apr 24, 2014 10:05 am
Contact:

State of the Sanqui Randomizer

Post by Sanqui » Mon Oct 17, 2016 8:36 pm

Hi! I made the Online Pokémon randomizer like two years ago at this point. Unfortunately, I haven't brought myself to work on it in a long time, which I feel really bad about. If anybody is perhaps interested in taking over or maintaining it, here's everything you need to know.

State of the Randomizer

Sadly, when I stopped working on it, the randomizer was left in a direly unfinished state. There are a few bugs that were never fixed, and the structure isn't sustainable.

What I had planned was create a rewrite with a whole new architecture that would make it easy to extend the randomizer, but sadly I never got too far.

I simply don't have the time or will to create the new version at the moment.

In the short term, it might be worth shaping up the current version, but I'd really love to see the rewrite happen... anyway, here's a description of how current randomizer works in all, in case you're interested in working on it. I'm going to cover every part of the randomizer.

The data

Pokémon sprites
Although only graphics, the sprites are arguably one of the most important things about the randomizer, because without them, it would be no fun to play at all.

What stood at the initial resource for the randomizer's sprites was a sheet of gen 1 styled Pokémon sprites that was created or collected by /vp/. Since it claimed anything can be done with the sheet without credit, I used it. Only later I learned that some sprites were put on it without the knowledge of their authors.

The sprites were closer to "shitty MS Paint" style than "Gen 1 style", so me and my friend Behold3r started working on replacing them with better sprites available on DeviantArt and other sites - always making sure to ask for permission from the artist. I believe about 2/3 of the sprites were replaced. I also drew some of the new sprites myself :)

The /vp/ sheet was also lacking Gen 6 Pokémon entirely, so we there was a bit of a team effort to draw new sprites for the randomizer.

Unlike the front sprites, there wasn't a good collection of backsprites. A little over half Pokémon got backsprites until now, the ones that don't have the frontsprite, cropped to the top right corner.

I always put a lot of value in making the game look "authentic", so to speak. I wanted sprites to be consistent to some extent with the original gen 1 look, which is why I kept the original palettes, backsprite sizes, and forbade the use of gen 2 sprites. Some guidelines for new sprites could be made.

I've been asked over the years to give away the sprites. I've been kind of dodging that issue, not for being selfish, but due to the vague licensing most of them have. Some are leftovers from the /vp/ sheet, with authors unknown, and for many, I've gotten exclusive license for randomizer usage, and no more. So that's probably how it's going to stay.

My friend, Behold3r, has a sheet with all the sprites, with one layer per author, and used to export it after each update. I can get it from him.

The shell script, convert_sprites.sh, downloads the sprite sheet, splits it up, generates backsprites, palettes, and compresses the sprites for use in the disassembly. It uses imagemagick. It requires gfx.py and pic.py from pokemon-reverse-engineering-tools to be in the same directory. Possibly old versions too.

Overworld sprites
My randomizer features randomized (actually shuffled) overworld sprites, which is certainly a fun feature. What most people probably don't realize is that about half the sprites in Red and Blue actually lack walking variants. All of those were done for my randomizer at some point by Reimu. They're in gfx/ow_sprites/ and probably end up in the ROM by some terrible black magic.

Pokédex

The Pokédex data is stored in a file called minidex.yaml. It was exported as a one-off from the open-source pokedex[1] database of veekun.com at the time of Pokémon X and Y, and uses learnsets from then. I will find the script that exported it. It should be possible to update to OR/AS immediately.

Music

The new songs (like gen 4 demixes) were mainly arranged by FroggestSpirit. He has done many more songs since then, I just haven't added them to the randomizer. The songs need to be in pokered-randomizer, and can be gotten from CrystalComplete, but check with FroggestSpirit. They work with the Gen 2 sound engine.

data/song_sources.yaml contains information on where each song came from.

data/songs.txt contains possible song substitutions for each song from the original game.

Other data

There's some other data that was mostly done as one-offs. It should be self-explanatory, let me know if you're missing something though.

The code

Now this is scary!

The randomizer could be split roughly into four major parts... The pokered-randomizer disassembly, the PokemonRed randomizer class, the randomizer code, and the website. frontend. Let me go over these.

pokered-randomizer
pokered-randomizer is an outdated fork of pokered with new features used in the randomizer. Contrary to logic, the ROM is built *first*, and the randomizer script works on the resulting binary. This is probably the biggest Achilles' heel. It allowed the randomizer to generate a ROM pretty fast, but the actual process of modifying data in the binary is extremely error prone...

I'm going to try and list the major changes done in this fork.

* Pokémon and Trainer IDs were successfully decoupled, meaning the game supports up to ~253 Pokémon (for the sake of simplicity, I went with 251). This has been done many times over and could probably be redone easily and better.
* The sound engine from pokecrystal was backported. This wasn't a trivial task, but not exactly complicated either. It allowed for use of real cries for Gen 2 pokémon, as well as all the gen 2 music, and new music that used the format. Danny-E has reproduced my commits against clean pokered at some point, as well as done some further fixes which I'm not sure if I cherry-picked.
* The sound engine gained support for pitch transposition (lol)
* Around 40 new moves were added, as well as old moves having their values changed to current values. This involved adding some new move effects, as well as animations for each of the new moves. This change is optional and set with a bitflag by the randomizer.
* Instant text was added, and toggled by a bitflag.
* Erratic and Fluctuating EXP groups were added.
* Probably more misc. stuff, as well as stuff being moved around for various reasons.

pokered-randomizer is included under the randomizer as a submodule. It needs to be bumped when updated.

PokemonRed
The randomizer contains a Game class. The class represents a game able to be randomized. PokemonRed is the only relevant subclass here. An instance of PokemonRed is a Red ROM, ready to be randomized by method calls, such as opt_starter_pokemon(), opt_wild_pokemon(), etc. You don't generally want to call those manually. Instead, you set the options in the choices dictionary, then run the produce() method, which respects priority. Sigh... Okay.

So when you look at the PokemonRed class, you'll see a few sections. First, a few constants relevant to the randomizer are defined. What follows is the form used by the website. It's defined using wtforms. This form is important, because the input names are used directly as methods on this class, and the option is passed as the argument. So even the possible parameters are defined here.

Afterwards, a lot of constants are defined through various ways - hardcoded, calculated from symbols, or even read from the ROM at start time. Honestly, it's pretty awful stuff.

Next, after auxillary methods, the methods for each randomization option are defined. Each of these methods begin with opt_ and perform their operation directly on the ROM in the works. Sometimes, things need to be done in a certain order, such as, if certain moves are banned (like Dragon Rage), that needs to be set before Pokémon learnsets or TM contents are generated. This is done using the .layer variable on the methods themselves.

The nastiest method here is opt_game_pokemon(), which is responsible for putting all the new Pokémon in. It actually works on an evolutionary family basis, pulling in family after family until the dex is full. (I fear that the routine at the heart of this is broken and may result in an infinite loop, but I haven't seen it happen yet.) Then it writes in the Pokémon data proper, base data, learnsets and evolutions, pics, etc.

There's of course many other routines here, some of them pretty scary in how they write in data without any real bounds checking and such.

randomizer
The randomizer, at the higher level, is pretty straightforward. It does some sanity checking on the options and has some methods to make working with the ROM easier. It also defines the games, which, besides Pokémon Red, happen to include Pokémon TCG and Telefang. Those are, however, incredibly basic in what they do, compared to the monster that is Pokémon Red. The idea was, of course, to support many more games over time, but it never became relevant.

app
The app is a dynamic website coded with Flask. There's some javascript in the frontend, written with jQuery (I swear that in 2014, it was still relevant), mainly to handle the forms and do an AJAX request for the ROM.

The website handles filename sanitation, as well as minute long cooldowns for IPs (too short? you decide). That's about.. it, really. I had it deployed with uwsgi_python behind nginx.

The Future of the Randomizer
I'm hesistant to straight up open source the whole thing. I'm not proud of it. It's not well written. I don't really want people to use it. But there's no way I'll be working on it myself any further.

If anybody wants to give a shot at fixing the few bugs or adding some features it direly needs, good luck... I'll be giving repo access to anybody who asks.

I've learned from the way I've done the randomizer and I hope that with this overview, you've learned something as well.

My plans for the actual replacement of the randomizer were, of course, grandiose. It wasn't meant to happen. But here's an overview of what I had planned, and what I'd still like to see one day.

Crowdsprite
Crowdsprite was a website which was to hold all the user-contributed data for the randomizer. This is primarily sprites. Gone would be coordinating with people over Skype, instead everybody could instantly see what sprites are done, which need replacement, and which are missing. Sprites could be validated for dimensions and the color palette, and replacements could be done and approved easily.

The plan was to extend Crowdsprite beyond sprites. People could submit text (e.g., for trainer names), as well as some more dreamy things, like maps or fakemons. It was fun to think about.

Randomizer 2.0
The next version of the randomizer would be tightly coupled with the relevant disassembly, building a ROM for each randomization request anew. Randomization would still be done in a Python script, which would export a huge list of constants. I was working on array support in rgbds, so that various stuff could be exported as lists, and easily iterated over in the source code. Could make do with just constants, though, it'd still be much better than modifying the binary.

Frontend
The frontend is easy, of course, but the current version isn't optimized for mobile at all. So the important bit here would be to make it responsive. The inputs could be made nicer or more dynamic, but that's not a priority.

Future games
Pokémon Crystal was the next step. With Crowdsprite under my belt, I figured sprite collection would be a pleasant activity. I also had some ideas for other games, like MMBN, but I'm sure anybody could come up with a ton of games they'd like to see randomized. The idea was to make it fairly easy to add new games, though.

What can be done?
Is there any interest in fixing the current version of the randomizer? Working on a new one? I'm sorry, but I can't do the lion's share of the work like I had planned.

As I've said before: if anybody with coding skills wants to take over the project, I can offer advice and consultation. It doesn't have to be in Python. I'd just love to see it happen somehow, and I'm sure there are many other people who would.

Feel free to ask me anything or brainstorm ideas. Also feel free to ask for access to the repo.
Last edited by Sanqui on Wed Aug 09, 2017 10:13 am, edited 2 times in total.
I am the person who has ruined Gold Glitched forever and seconhandedly, Red NSC too
Ask me to make a dumb romhack for you!
Image

Jul, a board.

Sanqui
Jr. Trainer
Posts: 90
Joined: Thu Apr 24, 2014 10:05 am
Contact:

Re: State of the Sanqui Randomizer

Post by Sanqui » Wed Nov 09, 2016 2:11 pm

Hey hey look I've released the source.
I am the person who has ruined Gold Glitched forever and seconhandedly, Red NSC too
Ask me to make a dumb romhack for you!
Image

Jul, a board.

FantomeSolire
Preschooler
Posts: 3
Joined: Mon Apr 24, 2017 2:06 am

Re: State of the Sanqui Randomizer

Post by FantomeSolire » Mon Apr 24, 2017 2:10 am

I know you don't have the will to continue the Randomizer, and I'm not telling you that I think you should or shouldn't continue it. I do know that there are a lot of people who really enjoy your Randomizer even though it's in a incomplete state, and I'm also thanking you for doing this for do long.

ClonaClox9999
Preschooler
Posts: 1
Joined: Wed Jun 21, 2017 12:44 pm

Re: State of the Sanqui Randomizer

Post by ClonaClox9999 » Wed Jun 21, 2017 12:45 pm

So how do I flipping PLAY the game??

FantomeSolire
Preschooler
Posts: 3
Joined: Mon Apr 24, 2017 2:06 am

Re: State of the Sanqui Randomizer

Post by FantomeSolire » Wed Jun 21, 2017 1:53 pm

You need to use a GB emulator or a GBC emulator to play any roms you create

Cosmic_Extremo
Preschooler
Posts: 1
Joined: Wed Jun 21, 2017 6:17 pm

Re: State of the Sanqui Randomizer

Post by Cosmic_Extremo » Wed Jun 21, 2017 6:21 pm

Hey , I have played your rom hack and I just wanna ask you if you can come back to this project ... I don't know much about coding and all , but I can try to provide to you sprites of Generation 7 pokemon ! In fact , I have actually made some sprites myself in the past. -Cosmic_Extreme

PLEASE REPLY EVEN IF YOU DON'T PLAN TO START WORKING AGAIN

knuxyl
Preschooler
Posts: 5
Joined: Thu Jun 29, 2017 2:24 pm

Re: State of the Sanqui Randomizer

Post by knuxyl » Thu Jun 29, 2017 2:33 pm

Where is the graphics for the pokemon? I searched through the sources and only found the originals in the fork of pret. I watch 360chrism and shenigans race on these randomizers and I can program and would like to continue your work.

I would just do it with the official pret source with a lua program. embedding it in a site would be the hardest part for me honestly, but the similarity between python and lua could make it easier since u already did that part.

Do you have any kind of instant messenger like facebook or something? I see that you are actually modifying the compiled rom, but I'm not sure how you managed to include the pokemon crystal sound engine. I'll probably have some more questions too.

I'm mostly interest in the sprites. I've been doing that stuff since I was like 13 and would like to finish the set to complete the pokedex.

thanks

Sanqui
Jr. Trainer
Posts: 90
Joined: Thu Apr 24, 2014 10:05 am
Contact:

Re: State of the Sanqui Randomizer

Post by Sanqui » Thu Jun 29, 2017 6:38 pm

Hi knuxyl, thanks for your interest!

As I mentioned in the original post, the sprites are downloaded by the convert_sprites.sh script in the online-randomizer repository. I only ask that when you use the sprites you give credit to all the artists I list on my site.

I've done many changes to the original pokered repo, including expanding the Pokédex to 251 Pokémon and the Crystal sound engine. This is all in the pokered-randomizer repo. However, there's also plenty of bugs, and the repository is outdated against upstream pokered. Other people have done similar hacks and recently dannye has reproduced my crysaudio work on a fresh repo.

I don't use Facebook. I can be best contacted at my email (gsanky@gmail.com), or Telegram (@Sanqui), or Discord (Sanqui#3248). I'll try to answer all questions you might have.
I am the person who has ruined Gold Glitched forever and seconhandedly, Red NSC too
Ask me to make a dumb romhack for you!
Image

Jul, a board.

knuxyl
Preschooler
Posts: 5
Joined: Thu Jun 29, 2017 2:24 pm

Re: State of the Sanqui Randomizer

Post by knuxyl » Fri Jun 30, 2017 6:40 am

I've got the working directory set up with all the sources but it seems the randomizer races I watch don't use yours anymore and I can't seem to figure out which one they are using. The one they use includes a instant text box, like you can basically skip all the text, unlike the instant text option where a makes the text instantly appear.

Sanqui
Jr. Trainer
Posts: 90
Joined: Thu Apr 24, 2014 10:05 am
Contact:

Re: State of the Sanqui Randomizer

Post by Sanqui » Fri Jun 30, 2017 6:52 am

knuxyl wrote:
Fri Jun 30, 2017 6:40 am
I've got the working directory set up with all the sources but it seems the randomizer races I watch don't use yours anymore and I can't seem to figure out which one they are using. The one they use includes a instant text box, like you can basically skip all the text, unlike the instant text option where a makes the text instantly appear.
Likely Dabomstew's Universal Pokémon Randomizer.
I am the person who has ruined Gold Glitched forever and seconhandedly, Red NSC too
Ask me to make a dumb romhack for you!
Image

Jul, a board.

knuxyl
Preschooler
Posts: 5
Joined: Thu Jun 29, 2017 2:24 pm

Re: State of the Sanqui Randomizer

Post by knuxyl » Sun Jul 02, 2017 5:59 am

Sanqui wrote:
Fri Jun 30, 2017 6:52 am
knuxyl wrote:
Fri Jun 30, 2017 6:40 am
I've got the working directory set up with all the sources but it seems the randomizer races I watch don't use yours anymore and I can't seem to figure out which one they are using. The one they use includes a instant text box, like you can basically skip all the text, unlike the instant text option where a makes the text instantly appear.
Likely Dabomstew's Universal Pokémon Randomizer.
Ok, yeah I saw that before but it doesn't have newer generation pokemon like yours does. I'm almost done batch processing the sprites from every pokemon game to the format that red can use. Not all look good, but I made a graph so I can select the best ones. If I continue this, I'll probably end up randomizing the sprite used as well.

Anyone can view the progress here
https://www.pokecommunity.com/showthrea ... ost9692599

Sanqui
Jr. Trainer
Posts: 90
Joined: Thu Apr 24, 2014 10:05 am
Contact:

Re: State of the Sanqui Randomizer

Post by Sanqui » Sun Jul 02, 2017 11:06 am

knuxyl wrote:
Sun Jul 02, 2017 5:59 am
Ok, yeah I saw that before but it doesn't have newer generation pokemon like yours does. I'm almost done batch processing the sprites from every pokemon game to the format that red can use. Not all look good, but I made a graph so I can select the best ones. If I continue this, I'll probably end up randomizing the sprite used as well.

Anyone can view the progress here
https://www.pokecommunity.com/showthrea ... ost9692599
I really don't like those conversed sprites. The outlines are very wonky, small Pokémon are disproportionally large, there is little dithering and texture. Because they are all in different styles it will be aesthetically very unplesaing if you choose random ones.

Friendly reminder that I ask again that you do not use the sprites from my randomizer without giving credit to the artists, who are listed on the randomizer page. I would also prefer if the sheet by itself wasn't posted because otherwise it'll get reposted everywhere without attribution.
I am the person who has ruined Gold Glitched forever and seconhandedly, Red NSC too
Ask me to make a dumb romhack for you!
Image

Jul, a board.

knuxyl
Preschooler
Posts: 5
Joined: Thu Jun 29, 2017 2:24 pm

Re: State of the Sanqui Randomizer

Post by knuxyl » Mon Jul 03, 2017 1:33 am

Yeah I'm aware of the bad sprite issue, I wouldn't be randomizing those until they get fixed up, but like pikachu from black/white 2 looks perfect, and so do many others. It's just to get started on making the sprites. I'm going to be doing the same thing to the backsprites and I expect those to turn out much worse, but again, this is just to get started. Actually working on the sprites will be much later, I'm going to have to pick and choose which ones to use.

Apparently you aren't aware of this, but these sprites have been floating around the internet for quite some time already
https://www.reddit.com/r/gaming/comment ... e_sprites/

I'm not taking down the montage. I'll see if I can put the credits within the montage itself This is a grey montage when a full color is already everywhere.

better?
Image
Last edited by knuxyl on Mon Jul 10, 2017 12:55 pm, edited 1 time in total.

knuxyl
Preschooler
Posts: 5
Joined: Thu Jun 29, 2017 2:24 pm

Re: State of the Sanqui Randomizer

Post by knuxyl » Mon Jul 03, 2017 6:20 am

This is some examples of newer generations sprites being touched up (i tested 2 of the worst ones after the filter)

This is what they looked like after the filter
ImageImage

After being touched up and put in game

ImageImage

I know they are really generic. I really like the art style of your custom sprites and some from the original games (especially gyardos in yellow) but it would take too long to complete this art style for every pokemon. Like Groudon from your sheet is awesome and fits into gen 1 style. I'm really just doing this to get bases for the newer gens. I already have every single pokemon in gen 1 format, I just need to touch them up now, which really shouldn't take long considering how long dragonite and scyther took.

FantomeSolire
Preschooler
Posts: 3
Joined: Mon Apr 24, 2017 2:06 am

Re: State of the Sanqui Randomizer

Post by FantomeSolire » Mon Jul 03, 2017 1:44 pm

Those look amazing compared to the original sprites.

Post Reply

Return to “Randomizers and Rom Hacks”

Who is online

Users browsing this forum: No registered users and 2 guests