Opened 3 years ago

Last modified 2 years ago

#3072 accepted task

setup vm1

Reported by: sa2ajj Owned by: sa2ajj
Priority: major Milestone: sys - other
Version: Keywords: ansible
Cc:

Description

We seem to have VM1 running (at least, it responds to a ping).

I wonder what is the current situation with it?

on IRC tardyp expressed his view on how things should be regarding nine in general.

Change History (21)

comment:1 Changed 3 years ago by sa2ajj

vm1 is also a FreeBSD machine.

Last edited 3 years ago by sa2ajj (previous) (diff)

comment:2 Changed 3 years ago by dustin

I have a login. I just ran

https://github.com/buildbot/buildbot-infra/pull/22

on the host so everyone should have access now.

comment:3 Changed 3 years ago by verm

The goal for VM1 is to do continuous testing of Buildbot in our 'tier1' operating systems -- ones that we test on after every commit. This would be at least:

  • Windows
  • OS X
    • We have money to purchase a mini, 2 SSDs and 16GB of memory as by license you can only emulate OS X on a mac.
  • FreeBSD
  • Linux (Ubuntu, Debian?)

After each commit we would:

  1. Reset VM back to a snapshot
  2. Install dependencies
  3. Build Buildbot
  4. Install master
  5. Install a slave
  6. Launch both
  7. Run a test build

These machines should be kept up to date as possible. The purpose of these tests is to ensure a user experience.

We would also have persistent VMs to run slaves off of to keep everything in-house. This keeps away any issues of slaves going down or outdated dependencies and lets us control our test environment.

These are as much goals as a plan. I would skip #1 and #2 above initially.

comment:4 follow-up: Changed 3 years ago by sa2ajj

My question was about the tool you have in mind to achieve this goal.

comment:5 Changed 3 years ago by sa2ajj

A copy of my mail:

We have a ticket (this ticket) regarding setting up vm1. However I could not find any information about what is the plan/idea on how to setup those virtual machines. Is bhyve the expected way or some other virtualisation system? Amar, could you please comment on the ticket?

comment:6 in reply to: ↑ 4 Changed 3 years ago by verm

Replying to sa2ajj:

My question was about the tool you have in mind to achieve this goal.

Yeah, sorry divided attention at the moment! Here is my email response:

Well, I was planning on using VirtualBox? to handle Windows and Bhyve for *BSD and Linux. Windows support should be here for bhyve as well as Illumos in the future so we could drop VirtualBox?.

comment:7 Changed 3 years ago by skelly

We should also upgrade it to 10.1 as there are a number of fixes and improvements to bhyve. I read it was usable in 10.0, but a little rough still.

comment:8 Changed 3 years ago by sa2ajj

I invested a bit of time on vm1 for its primary purpose:

  • bhyve seems to be a bit too complicated (grub-bhyve + bhyve)
  • VirtualBox seems to be much easier

I found an excellent walk through, which somewhat mirrors the build script in vagrant Debian box creation.

I used the latter a bit more than a year ago and would like to use it for pre-seeding logic. I'd prefer to go with Debian/testing though (stable enough and recent enough; now that Debian is in freeze, it's very stable indeed).

comment:9 Changed 3 years ago by sa2ajj

Per quick exchange on IRC: I do not mind using bhyve as long as it can be fully scripted.

comment:10 Changed 3 years ago by sa2ajj

A potentially useful stuff (at least, for reviewing): https://github.com/michaeldexter/vmrc

comment:11 Changed 3 years ago by sa2ajj

Upgrade -- #3146.

comment:12 Changed 3 years ago by sa2ajj

  • Owner set to sa2ajj
  • Status changed from new to accepted

Well, upgrade is now done.

I'll spend some quality time with bhyve...

comment:13 Changed 3 years ago by sa2ajj

This time the installation of a Debian system under bhyve was pretty much straightforward.

I need to document a few bits related to this + update our playbooks to bootstrap this Linux environnment.

comment:14 Changed 3 years ago by sa2ajj

Primary interface works just fine: tap0 -> bridge0 -> lagg0.

Secondary interface does not: tap1 -> bridge1 -> vlan0 is setup, however pings work for the vlan0's address of vm1. I probably miss something.

comment:15 Changed 3 years ago by verm

I will help with this as soon as I can I'm in a meeting give me about an hour or so.

comment:16 Changed 3 years ago by sa2ajj

dmesg shows

bridge1: error setting interface capabilities on vlan0

which is not present for bridge0.

comment:17 follow-up: Changed 3 years ago by sa2ajj

For the record, the interfaces are created like this:

sysctl net.link.tap.up_on_open=1
sysctl net.inet.ip.forwarding=1

ifconfig tap0 create
ifconfig bridge0 create
ifconfig bridge0 addm lagg0 addm tap0 up

ifconfig tap1 create
ifconfig bridge1 create
ifconfig bridge1 addm vlan0 addm tap1 up

and later used like this

...
      -s 2:0,virtio-net,tap0 \
...
      -s 4:0,virtio-net,tap1 \
...

inside the Debian system they are configured like this:

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
	address 140.211.10.227
	netmask 255.255.255.128
	network 140.211.10.128
	broadcast 140.211.10.255
	gateway 140.211.10.129
	# dns-* options are implemented by the resolvconf package, if installed
	dns-nameservers 140.211.166.131
	dns-search buildbot.net

allow-hotplug eth1
iface eth1 inet static
	address 192.168.80.227
	netmask 255.255.255.0
	network 192.168.80.0
	broadcast 192.168.80.255

comment:18 in reply to: ↑ 17 Changed 3 years ago by verm

Replying to sa2ajj:

allow-hotplug eth1
iface eth1 inet static
	address 192.168.80.227
	netmask 255.255.255.0
	network 192.168.80.0
	broadcast 192.168.80.255

This needs a vlan tag of 80 in order to work.

comment:19 Changed 3 years ago by sa2ajj

so it should be (according to the docs)

iface eth1.80 inet static
...

comment:20 Changed 3 years ago by sa2ajj

well, it looks like if I configure it like this:

iface eth0.80 inet static
...

then it works.

comment:21 Changed 2 years ago by dustin

  • Milestone changed from sys - on-bb-infra to sys - other
Note: See TracTickets for help on using tickets.