Opened 6 years ago

Closed 6 years ago

#3034 closed task (fixed)

manage jails with Ansible

Reported by: dustin Owned by: sa2ajj
Priority: major Milestone: sys - on-bb-infra
Version: Keywords: ansible
Cc: verm

Description

Each service host should manage the jails on that host using Ansible.

Change History (13)

comment:1 follow-up: Changed 6 years ago by dustin

  • Cc verm added

At the moment, every jail has two IP addresses assigned (internal and external).

We should consider whether that's necessary. Many jails will only make outgoing connections, and could just as well be NATted or only have an internal address. Such jails would include buildslaves for the metabuildbot and the jail to run maintenance crontasks like the weekly-update script.

We might also consider using name-based virtual hosting on some of our web properties, with the service host acting as a frontend and proxying to the appropriate jail via localhost. That would save us from wasting an IP address on every domain name we set up.

comment:2 in reply to: ↑ 1 Changed 6 years ago by verm

Replying to dustin:

At the moment, every jail has two IP addresses assigned (internal and external).

Some services like database access and NFS don't need public IPs. Also I was eventually planning to install SSH on every jail for easier access.

We should consider whether that's necessary. Many jails will only make outgoing connections, and could just as well be NATted or only have an internal address. Such jails would include buildslaves for the metabuildbot and the jail to run maintenance crontasks like the weekly-update script.

That sounds too complex for our simple setup. We have the IP addresses alotted to us if we don't use them they'll sit there.

We might also consider using name-based virtual hosting on some of our web properties, with the service host acting as a frontend and proxying to the appropriate jail via localhost. That would save us from wasting an IP address on every domain name we set up.

If the only reason is to save an IP that sounds like a lot of work to me. What will we do with them once they're free?

comment:3 Changed 6 years ago by sa2ajj

  • Type changed from enhancement to task
  • Version 0.8.9 deleted

comment:4 follow-up: Changed 6 years ago by dustin

Do we have the whole /25? That is quite a few IPs, it's true :)

comment:5 in reply to: ↑ 4 Changed 6 years ago by verm

Replying to dustin:

Do we have the whole /25? That is quite a few IPs, it's true :)

RTEMS has a /25. I allocated a /27 to Buildbot out of that. However if we need more IPs we can have them so can RTEMS it was just a starting figure as I didn't see both projects using more than a /25.

If (and that's a huge if) we run out of addresses we can do the NAT trick then. I loathe to put everything behind a SPF if we don't have to. It's a lot easier to maintain if each jail is self-hosted.

I do want to get Varnish up for trac though with a simple plugin to invalidate various sections depending on a ticket update, wiki update or commit. I was going to put that instance inside the trac jail itself.

comment:6 Changed 6 years ago by sa2ajj

If I understood documentation right, there's an entity called 'base jail'. We probably should install all mandatory stuff to this jail to save space and simplify maintenance.

comment:7 Changed 6 years ago by verm

Yes, this is how it works. They were setup using ezjail you can update all jails by modifying the basejail which is mounted as read-only inside each jail.

comment:8 Changed 6 years ago by sa2ajj

skelly just mentioned on IRC that it's only about the base system, while I had in mind stuff that would go to /usr/local

comment:9 Changed 6 years ago by dustin

  • Status changed from new to assigned

The approach we're taking now is that each "host" (jail or hardware) runs Ansible on its own in a crontask. So all that remains to do here is to bootstrap each jail from its service host.

I don't know what the best approach is to do that, so I'm unassigning from myself.

comment:10 Changed 6 years ago by sa2ajj

  • Owner set to sa2ajj
  • Status changed from assigned to accepted

Another option to consider (and it looks very appealing to me) -- qjail.

I'll postpone its implementation until pgsql jail is about to be done (qjail provides means for as fine grain control as jail itself unlike ezjail).

comment:11 Changed 6 years ago by sa2ajj

Amar, what were the problems you experienced with ezjail under FreeBSD 10.0?

comment:12 Changed 6 years ago by sa2ajj

I think we have a decent way to manage jails at the moment.

Further evaluation of qjail is to be done as part of #3137.

(I'd still like to hear about problems with ezjail.)

comment:13 Changed 6 years ago by sa2ajj

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.