Reducing single points of failure

Community
#1

This was originally posted by me in the dev chat, so context might be lacking a bit.

We should improve our single-point-of-failure or bus-factor situation. From the top of my head there are two things with a bus-factor of one (only Sheogorath has access to the demo instance and only IndieHost have access to our Discourse). I’d like to ensure that all maintainers have access to these pieces of “critical infrastructure” e.g. by hosting them on a VM every maintainer has access to.

Why don’t we just buy a VM at https://www.hetzner.com/cloud and host the Forum and the Demo there? You can share access to the admin console to other people, which would solve the access-problem neatly.

#2

I’m not to sure we need to move from IndieHost away, if more peope then @sheogorath are able to talk to them tbh, but if the majority wants this, I won’t resist this change.

#3

For indiehosters, I usually contact them using their regular contact address:

https://indiehosters.net/contact/

Nothing special there :slight_smile: @unteem also has an account in the forum if there is anything wrong, but usually is not overly active. And since you can proof that you have admin rights in the Forum, I don’t think there is much of a SPoF around.

For the demo instance, I’ll talk about the details alter, when I have more time to write up on it.

#4

Hey there.

I’ll let you decide what you feel is best for you. I would not say we are a single point of failures, like with any SaaS provider while you do not have root access to the server we give you admin access to the discourse. Well you are not totally independent, it has to go trough us for plugins deployments but we try to be as reactive as possible :slight_smile:

So yep we would be happy to keep hosting the forum, we are happy to support Hedgedoc but if you feel its best to move, let us know we will give you everything you need to migrate.

Actually I would be more eager to do more for Hedgedoc than less. We could help for CI/CD and demo deployments for instance (a bit busy at the moment but we could discuss about it).

PS: our contact email is the way to go, goes through our ticketing system. We might come here from time to time but that is not guaranteed.

4 Likes
#5

And i’m the other SpoF of Indie Servers :wink:

And also if you need a VM at hetzner, we’d be happy to finance it.

Feel free to ask, we love HedgeDoc, and we are happy to contribute the way we can :slight_smile:

(I’ll migrate the forum to the new domain this Friday afternoon)

cheers!

2 Likes
#6

IMHO discourse is less of a SPoF as long as we can reach out to you at IndieHosters and you’re reacting on time.
Also hosting discourse is an … exhausting task and if you want to do this I wouldn’t hold you back. :sweat_smile:

A vm at hetzner, netcup or somewhere else would be nice, because it’s always good if more than one person has access critical infrastructure. Especially in emergency situations. I would also prefer this, because it’s independent from personal setups.

Also it would be nice to collect our domains at one domain broker. Right know our domains are distributed to many spof people. This could be possible with an account at hetzner or namecheap.

#7

Also imho a central place for our domains is more important than the VM.

2 Likes
#8

The main problem residing around the demo instance, is that my infrastructure is not intended for multi-tendency and therefore can not ensure the level of separation and security that would be required to allow anyone else shell access to those machines.

I’m currently exploring what easy to use ways I can provide to allow access by community members to work on the demo instance themselves, but so far, the options turn out to be either overly complicated or compromising on security.

If this is intended to be resolve this by moving the content to a separate machine, I’m open for that, but I would step back from maintaining this setup as it would probably not fit overly well in my quite opinionated setup.

Note: The risk of the Bus factor is already limited as I ensured that at least one other person who is also aware of the HedgeDoc community can get access to the entire infrastructure if something happens to me.

#9

So let’s think about that topic:

Do we have more domains? :smiley:

The easiest thing would probably be to move both registration and DNS of all domains to Cloudflare (they support multi-user accounts). If we don’t want to concentrate everything at Cloudflare, we could also use e.g. namecheap as a registrar and Cloudflare for DNS, but I’d like to avoid having to juggle with more than two additional accounts.

#10

Another thing that came to my mind: What about our social accounts? @sheogorath I think currently only you can manage them?

#11

For social media accounts, as written in the description of them, they are managed by me and @Amolith. We can definitely further distribute that, but so far people don’t seem overly keen to produce content.

#12

The idea behind splitting namecheap and Cloudflare is to reduce risk. Not too many eggs in one basket. Cloudflare API keys are floating around to use DNS challenges etc. therefore the separation.

The access to namecheap was granted for the disaster case where a full recovery appears to be required, allowing you and @amenthes to move the DNS servers to a different account.

Before we can move DNS to another account, I have to solve some setup related assumptions, to make sure certificate generation continuous to function without problems.

#13

To provide some more insight into possible services that are currently run in my Infrastructure but might be migrated to a separate one:

  • (Inbound) E-mail management for Discourse
  • Monitoring of the Demo Instance (external uptime, stats, etc.)
  • Logging, currently the centralized logging is also handled in the Infrastructure setup.
  • all redirect pages and domains
  • Backups and restore for all services

Maybe some further minor services I can’t think of right now.

#14

I’m happy with namecheap as broker and cloudflare as reverse proxy , especially, because we have international users.

(cloudflare shared access is only available on enterprise level).
edit: (okay i checked that again. seems like they unlocked that for free accounts, too)

#15

But well…

Namecheap for Domains
Cloudflare for DNS
(Hetzner for VM?)

That are all services that allow shared admin access (and security stuff like 2FA).
Sounds good to me.

#16

Just for adding details about the current situation:
The hedgedoc.net domain is currently registered (until 07/22) with porkbun.com. I’ll investigate whether porkbun allows multi-user per domain setups.
DNS is deployed via cloudns.net global anycast network. @davidmehren has (besides me) access to the DNS entries.