A glitchy but lovable microblogging server https://toot-lab.reclaim.technology/
Go to file
Brian Campbell 2d12948220
Fix idempotency when database writes are slow (#21840)
There is an idempotency key generated by clients when authoring a post,
and stored in Redis, to ensure that if a user or client retries posting
the same status, we don't get a duplicate.

Hachyderm.io has been experiencing some filesystem and database
performance issues, causing database writes to be slow. This can mean
that there are successful posts, but the reverse proxy returns 504
Gateway Timeout before the idempotency status has been updated; users or
clients who retry (such as Tusky which retries automatically, see
tuskyapp/Tusky#2951) can re-try the same post with the same idempotency
key before it has actually been recorded in Redis, leading to duplicate
posts.

To address this issue, move all of the database updates after the
initial transaction that creates the status into the
`postprocess_status!` method, so we can insert the idempotency key
immediately after the status has been created, significantly reducing
the window in which the status could be created but the idempotency key
not yet stored.

Note: this has not yet been tested; I'm submitting this PR for
discussion and to offer to the Hachyderm.io admins to try out to fix the
multiple posting problem.

Co-authored-by: Brian Campbell <brcampbell@beta.team>
2022-12-15 18:08:40 +01:00
.circleci Align everything to Node.js 16 (#22223) 2022-12-15 17:43:26 +01:00
.devcontainer Align everything to Node.js 16 (#22223) 2022-12-15 17:43:26 +01:00
.github Add hadolint as Dockerfile linter (#20993) 2022-12-15 15:57:17 +01:00
app Fix idempotency when database writes are slow (#21840) 2022-12-15 18:08:40 +01:00
bin Update Mastodon to Rails 6.1 (#15910) 2021-03-24 10:44:31 +01:00
chart helm: cleanup helm chart, now in mastodon/chart (#21801) 2022-12-09 06:36:29 +01:00
config Add brotli compression (#19025) 2022-12-15 17:07:36 +01:00
db Fix pre-4.0 admin action logs (#22091) 2022-12-06 23:38:03 +01:00
dist Fix nginx location matching (#20198) 2022-11-09 04:12:57 +01:00
lib Fix default S3_HOSTNAME used in mastodon:setup (#19932) 2022-12-15 16:38:51 +01:00
log Initial commit 2016-02-20 22:53:20 +01:00
public Embed js height fix (#22141) 2022-12-15 16:18:59 +01:00
spec Fix changing domain block severity not undoing individual account effects (#22135) 2022-12-15 17:45:02 +01:00
streaming Fix filters from other users being used in the streaming service (#20719) 2022-11-15 02:09:58 +01:00
vendor Increase files checked by ESLint (#9705) 2019-01-04 11:28:38 +01:00
.browserslistrc Add ES6 compatibility to browserslist (#18519) 2022-05-26 20:29:28 +02:00
.buildpacks Remove nodejs buildpack from buildpacks (#14364) 2020-07-20 09:48:10 +02:00
.codeclimate.yml Replace from Code Climate to Super-Linter (#18587) 2022-06-01 19:22:35 +02:00
.deepsource.toml Update .deepsource.toml (#15753) 2021-02-19 09:53:25 +01:00
.dockerignore Update .dockerignore/.gitignore for #16947 etc (#17099) 2021-12-06 18:04:04 +01:00
.editorconfig Add final newline to locale files (#2890) 2017-05-07 19:55:47 +02:00
.env.production.sample ip_cleanup_scheduler: Make IP and session retention configurable (#18757) 2022-07-07 03:14:28 +02:00
.env.test Bump webpacker from 4.0.7 to 4.2.0 (#12416) 2019-11-20 17:56:11 +01:00
.env.vagrant Add a default DB_HOST to .env.vagrant for enable the streaming (#14030) 2020-06-15 01:57:06 +02:00
.eslintignore Increase files checked by ESLint (#9705) 2019-01-04 11:28:38 +01:00
.eslintrc.js Replace from Code Climate to Super-Linter (#18587) 2022-06-01 19:22:35 +02:00
.foreman Replace sprockets/browserify with Webpack (#2617) 2017-05-03 02:04:16 +02:00
.gitattributes Add .gitattributes file to avoid unwanted CRLF (#3954) 2017-06-26 13:15:24 +02:00
.gitignore helm: cleanup helm chart, now in mastodon/chart (#21801) 2022-12-09 06:36:29 +01:00
.haml-lint.yml Added haml-lint and fix warnings (#2773) 2017-05-08 03:35:25 +02:00
.nanoignore Remove Storybook (#4397) 2017-07-27 22:30:27 +02:00
.nvmrc Align everything to Node.js 16 (#22223) 2022-12-15 17:43:26 +01:00
.prettierignore helm: cleanup helm chart, now in mastodon/chart (#21801) 2022-12-09 06:36:29 +01:00
.prettierrc.js Format JSON and YAML using Prettier (#17823) 2022-03-21 04:46:11 +01:00
.profile Add ffmpeg and dependent packages as well as LD_LIBRARY_PATHs (#3276) 2017-05-24 17:57:33 +02:00
.rspec Adding a Mention model, test stubs 2016-02-25 00:17:01 +01:00
.rubocop.yml linting: RuboCop update, config fixes (#20574) 2022-12-15 16:39:59 +01:00
.ruby-gemset Make Mastodon use its own gemset (#17858) 2022-08-13 15:44:34 +02:00
.ruby-version Bump Ruby version from 3.0.3 to 3.0.4 (#18028) 2022-08-15 04:39:58 +02:00
.slugignore Remove Storybook (#4397) 2017-07-27 22:30:27 +02:00
.yarnclean Replace from scss-lint to sass-lint (#10958) 2019-06-04 17:23:18 +02:00
AUTHORS.md Update AUTHORS.md (#20630) 2022-11-14 08:33:24 +01:00
Aptfile Heroku fix (#19807) 2022-11-05 22:57:58 +01:00
CHANGELOG.md Bump version to 4.0.2 (#20725) 2022-11-15 03:57:18 +01:00
CODE_OF_CONDUCT.md Add code of conduct from GitHub generator (#5674) 2017-11-13 17:28:55 +01:00
CONTRIBUTING.md Note that CircleCI auth may be required to run PR pipelines (#20371) 2022-11-11 02:54:02 +01:00
Capfile remove capistrano/faster_assets from Capfile (#2737) 2017-05-03 12:14:52 +02:00
Dockerfile Add hadolint as Dockerfile linter (#20993) 2022-12-15 15:57:17 +01:00
FEDERATION.md Add FEDERATION.md (#17029) 2021-11-23 00:15:31 +01:00
Gemfile linting: RuboCop update, config fixes (#20574) 2022-12-15 16:39:59 +01:00
Gemfile.lock linting: RuboCop update, config fixes (#20574) 2022-12-15 16:39:59 +01:00
LICENSE Fix #49 - License changed from GPL-2.0 to AGPL-3.0 2016-09-21 23:04:34 +02:00
Procfile fix: make Procfile compatible with herokuish (#12685) 2020-08-29 19:00:30 +02:00
Procfile.dev Set RAILS_ENV in Procfile.dev (#15502) 2021-01-07 09:30:12 +01:00
README.md Align everything to Node.js 16 (#22223) 2022-12-15 17:43:26 +01:00
Rakefile Initial commit 2016-02-20 22:53:20 +01:00
SECURITY.md Change e-mail in SECURITY.md (#20384) 2022-11-11 05:26:43 +01:00
Vagrantfile Update Ubuntu, Node versions, dependencies (#22075) 2022-12-07 00:13:14 +01:00
app.json Make enable_starttls configurable by envvars (#20321) 2022-11-10 21:06:21 +01:00
babel.config.js Remove dependency for @babel/plugin-proposal-class-properties (#16155) 2021-05-04 23:13:04 +02:00
config.ru Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
crowdin.yml Prevent translations from accidentally becoming completely different from source strings on Crowdin without translators noticing. (#17085) 2021-12-01 19:03:35 +01:00
docker-compose.yml Update docker-compose.yml (#19063) 2022-08-27 17:40:46 +02:00
ide-helper.js Add JS IDE helper (#13012) 2020-04-27 17:19:53 +02:00
jest.config.js Bump jest from 27.5.1 to 28.0.3 (#18280) 2022-05-04 14:06:49 +09:00
jsconfig.json Correct prettier calls for YML/JSON (#21832) 2022-12-03 12:05:35 +09:00
package.json Align everything to Node.js 16 (#22223) 2022-12-15 17:43:26 +01:00
postcss.config.js Increase files checked by ESLint (#9705) 2019-01-04 11:28:38 +01:00
priv-config TOR federation (#7875) 2018-06-26 20:34:12 +02:00
scalingo.json Make enable_starttls configurable by envvars (#20321) 2022-11-10 21:06:21 +01:00
stylelint.config.js Replace from Code Climate to Super-Linter (#18587) 2022-06-01 19:22:35 +02:00
yarn.lock Bump postcss from 8.4.19 to 8.4.20 (#22256) 2022-12-14 05:41:52 +09:00

README.md

Mastodon

GitHub release Build Status Code Climate Crowdin Docker Pulls

Mastodon is a free, open-source social network server based on ActivityPub where users can follow friends and discover new ones. On Mastodon, users can publish anything they want: links, pictures, text, video. All Mastodon servers are interoperable as a federated network (users on one server can seamlessly communicate with users from another one, including non-Mastodon software that implements ActivityPub)!

Click below to learn more in a video:

Screenshot

Navigation

Features

No vendor lock-in: Fully interoperable with any conforming platform

It doesn't have to be Mastodon; whatever implements ActivityPub is part of the social network! Learn more

Real-time, chronological timeline updates

Updates of people you're following appear in real-time in the UI via WebSockets. There's a firehose view as well!

Media attachments like images and short videos

Upload and view images and WebM/MP4 videos attached to the updates. Videos with no audio track are treated like GIFs; normal videos loop continuously!

Safety and moderation tools

Mastodon includes private posts, locked accounts, phrase filtering, muting, blocking and all sorts of other features, along with a reporting and moderation system. Learn more

OAuth2 and a straightforward REST API

Mastodon acts as an OAuth2 provider, so 3rd party apps can use the REST and Streaming APIs. This results in a rich app ecosystem with a lot of choices!

Deployment

Tech stack:

  • Ruby on Rails powers the REST API and other web pages
  • React.js and Redux are used for the dynamic parts of the interface
  • Node.js powers the streaming API

Requirements:

  • PostgreSQL 9.5+
  • Redis 4+
  • Ruby 2.7+
  • Node.js 16+

The repository includes deployment configurations for Docker and docker-compose as well as specific platforms like Heroku, Scalingo, and Nanobox. The standalone installation guide is available in the documentation.

A Vagrant configuration is included for development purposes. To use it, complete following steps:

  • Install Vagrant and Virtualbox
  • Install the vagrant-hostsupdater plugin: vagrant plugin install vagrant-hostsupdater
  • Run vagrant up
  • Run vagrant ssh -c "cd /vagrant && foreman start"
  • Open http://mastodon.local in your browser

Contributing

Mastodon is free, open-source software licensed under AGPLv3.

You can open issues for bugs you've found or features you think are missing. You can also submit pull requests to this repository or submit translations using Crowdin. To get started, take a look at CONTRIBUTING.md. If your contributions are accepted into Mastodon, you can request to be paid through our OpenCollective.

IRC channel: #mastodon on irc.libera.chat

License

Copyright (C) 2016-2022 Eugen Rochko & other Mastodon contributors (see AUTHORS.md)

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.