RainRatBot - Frequently Asked Questions

Are you going to open source your program or data set?

I'm not planning to release them. For the program, remember that I am using Clarifai for the recognition. Reading my code wouldn't tell you anything about AI. You would learn more about the Clarifai API from reading their docs than trying to understand what my code does.

If I released the data, there would be a lot of photos separated from any credit to their photographers. It wouldn't be fair to them to release it myself. Perhaps one day if I can't provide this service any more, I'll pass them on to a single successor, who I would ask they also keep them out of general distribution.

What if I don't want my fursuit recognized?

I'm pretty sure I'm be on solid legal ground have it recognized even if the owner doesn't want it, since the only thing it replies with is a public name; the users don't even see the photos used to train it.

But, I don't really want to spend CPU cycles and storage on someone not interested, so I'd remove someone who didn't want to be included.

I don't like it replying to my photos in a group that I'm in. Can you have it not do that?

It's up to the group owner whether they add the bot, and they can remove it if they choose.

Now, if you don't like it replying to your photos because its reply is wrong, I can help with that. Just report it to me (including the photo that triggered it), and I'll fix it in the next update. I don't know if it's giving wrong results unless someone reports it.

Just for fun - Tee KO ideas

Do you ever play Tee KO from Jackbox Games? If not, you can probably skip this post.

Ever not able to come up with good ideas for drawings or slogans?

You can message @RainRatBot with /getdrawings or /getslogans to get some ideas. They're randomly selected from a pool of ideas.

It's totally unofficial, so use as many of the ideas as you like, or none of them.

You can suggest new ones with /suggestdrawings or /suggestslogans

It's not intended for all the slogans to be funny on their own. Simple ones like "YEP" or "NOPE" the humour is expected to come the combination with the image. But if any are always terrible, you can message me. They're sourced from suggestions, so if any are more offensive than necessary, you can also message me about it.

Other uses of RainRatBot - Moderation

The fursuit identification bot is based on a Clarifai custom model, but Clarifai also has some pre-made models that are worth looking at.

The Moderation model stands out as something that could be useful as a Telegram bot. Owners of SFW (Safe For Work) groups could have a bot checking photos posted to the group, and trigger some sort of action if a photo meets some criteria. The model comes with the categories:

• explicit
• suggestive
• drug
• gore

Like the other models, these categories come with a confidence.

For instance this photo came with a confidence of 92.65% for "suggestive".

Something like that wouldn't trigger an automatic action from the bot in most groups. Right now, this is experimental. I have this running in an SFW group right now, but it only messages the owner and moderators of the group if it sees something above the threshold. I don't want to say the name of the group because I don't want people going to this SFW group just to test the limits.

In the future, the group owner might add RainRatBot, then specify a policy they'd like applied. For instance, they might say, "Delete any photo that is 'explicit' with 99%+ likelihood, then message the moderators if any of the categories are 90%+"

To get a feel for what the confidences and categories translate to for things that might be posted to your group, I created a group the bot treats specially. It will run the photo past the different models and tell you what the results were. If you are a group owner, message me at @RainRattie to get the link. (only join if you are 18+ and ok with seeing adult material).

How do I add a fursuit to be recognized by the bot?

Machine Learning works from examples. It starts off with correctly-named examples, then it works out what features are relevant to tagging the photo with each name.

The most important need is a significant number of photos that are of the fursuit that is to be added.

By significant number, I suggest as a rule of thumb, at least 30. On the higher end, some of the fursuits have over 200 labelled photos, which also works.

The model is not directly crowdsourced, I upload the photos myself, after checking them. A couple options:

• You can upload them to me on Telegram. My user name is @RainRattie. To save the time of individually uploading each one, put them all in one ZIP file, and use the Send File function.
• Or, if you don't know about ZIP files, send them as photos to the bot. As mentioned in the other post, I do look at submissions. If I'm looking for a specific fursuit, I can grab the photos from the incoming files.

Then you have to tell me the name of the fursuit. If you messaged me the photos directly, you can tell me the name as you send it. If you sent the photos to the bot, also message me, but tell me what to look for in the incoming files.

Some suggestions:

• The only fursuit in the photos should be the one that you are adding.
• Backgrounds are ok; they can be busy, plain, light, dark, outdoors, indoors. In fact, please change up the backgrounds as much as possible.
• Change up the lighting. It doesn't mean you have to take photos that are super-dim or super-bright. Just that fursuits look quite different under sunlight, incandescent, and florescent lights.
• Change up angles and poses. For instance, if all your photos are straight-on of the face, and the model is trained on that, later when someone sends a photo of a side view, it's going to look totally different and going to be unrecognized.

How does the bot work behind the scenes?

The bot is based on two existing technologies, the Python Telegram Bot framework, and a Clarifai custom model.

The Python Telegram Bot framework connects to Telegram using the RainRatBot login, waits for a specific action (a photo), downloads the photo, and replies with the result.

The Clarifai custom model has two parts, it has a human-readable web interface that can be used to maintain and train the model.

Then there is the Clarifai API (Application Programming Interface), that RainRatBot can upload a photo, and receive a reply in a standard format that it can use.

The training of the model is based on correctly labelled images. Currently although I'm accepting feedback and photos for new submissions, it isn't directly crowdsourced. I look over the feedback and new photos, and upload new training data somewhere within a day to a couple weeks.

I also look over the submissions. If the submission is a fursuit I can identify, and I believe adding it to the training set would improve the model, I will add it to the training set.

• I look at the submissions.
• I do not log who sends the submissions.
• Do not use it if you do not want a human looking at the photo at some point, even anonymously.
• It might be possible to break the anonymity, forensically. I haven't looked into it, but don't rely on absolute anonymity.

Using RainRatBot in a Telegram Group

RainRatBot can also identify fursuits in photos posted to a Telegram Group.

From a group's description page, a user can choose "Add Member" (the exact location might vary from client to client), then click the "search" icon (the magnifying glass), then type "RainRatBot".

Choose it, and that's all. You do not have to give it moderator privileges. It is checking each photo posted to the group against its model of the different fursuits. There is one big difference between groups and privately messaging the bot. In a group, it will only reply if its confidence is above 90%. It keeps the bot from getting too chatty. I've seen active groups go for days or weeks without the bot saying anything, then when a user posts a fursuit photo they wished to know the name of when they weren't even thinking about the bot, they get a reply.

Make sure you have permission of the group owner before adding it. Since it downloads photos posted to the group, don't add it to a group where just downloading the photos would breach secrecy.

• Photos are downloaded and saved.
• The group name, and user name are not logged. I do not know which photo comes from which group. This may change if I need to track abuse.
• It does not download any other content besides the photos. It currently does not have that functionality. This will not change. The only way I see that happening is if a group owner requests to have some custom functionality written for their specific group, and it would hardcode it to that specific group.

How do I try it?

Currently, there is a limited way to give it a try. I hope to have more ways to use it in the future, but it currently works as a Telegram bot.

Telegram is an instant messenger app that is popular with furries. It supports private messages and groups, as well as a bot interface.

After downloading the Telegram client, the user can message the bot at @RainRatBot like they would message any other user. Unlike a user, the only things it will respond to are photos, and a few specifically formatted commands, like "/start"

What's going on here? Well the user sends their photo of this orange fursuit, using the usual Telegram features like they would send a photo to any other user.

Behind the scenes, my server downloads the message from Telegram, then analyses it uses a previously-created Machine Learning model. Then it replies:

Fursuit Name:
Sunny Dingo (Made Fur You) 97.89%

"Fursuit Name:" is included in all replies to photos.

"Sunny Dingo (Made Fur You)" is the name of the fursuit. It knows this information because earlier, photos of this same fursuit were labelled "Sunny Dingo (Made Fur You)" in the training set. (Made Fur You is a fursuit making company. All the fursuit names are labelled with their maker as well, first to give the maker proper credit, second because many fursuits have simple names, and it helps narrow down exactly which suit is being referred to.)

"97.89%" is a confidence level. Based on the information the model is created with, it is 97.89% certain that the label Sunny Dingo (Made Fur You) is correct.

You can give it a try with your own photos. Since you're going to the effort of sending photos, I programmed it to always reply to you with its best prediction, even if the confidence is very low. For instance, if you send a fursuiter that it doesn't know, it might reply with a guess of a fursuiter that it does know, but with a confidence of around 20%. If someone were using the results, they might only use results that were 80%, 90% or even higher confidence, but for this usage, since there's no harm in a wrong guess, it will always give you the result.

Starting off with some background

I'm working on a passion project for a group of people known as furries. The Furry Fandom in general are people who are interested in animals with human characteristics. You can read more about the fandom in general at Wikipedia.

Furries hold many conventions a year. Costumes known as fursuits are common, though the majority of furries do not own one. Many furries only relation with fursuits is to take pictures.

After the convention, often photographers want to identify the fursuit in their photos, to accurately describe them. (Many fursuiters do not speak in fursuit, so photographers often don't get the fursuiter's name.) Fursuiters often search social media after a convention, since that's their memory of their time in fursuit.

I'm looking for a way to bring the two together.

Machine Learning has advanced a lot in the past few years. Given a set of correctly labelled images, a computer can pick out relevant details, and create a model of what is relevant to correctly identify an image of the same thing in the future.

So that's my plan. Take a bunch of correctly labelled images of fursuits, and create a Machine Learning model that can name the fursuit in any new photo.

• Photographers can upload photos with fursuits they don't know. The model would instantly reply back with the name of the fursuit.
• Fursuiters may get a benefit out of this even if they don't hear about this service. More photographers will be able to identify the fursuits in their photos, and credit the fursuiter when they post to social media.