This week I had a "pleasure" of reimaging Exadata server to use OVM.
During this process we hit interesting problem – after the reimaging process there was no bridge at DOM0, corresponding to the client network – because of this, the OEDA sofware returned an error like this:
[ERROR ] More than two single interfaces are found in DOM0 that match criteria. Single interfaces; IP; Netmask, Gateway: ; 10.20.10.70; 255.255.0.0; 10.20.0.1
The error was produced by this script: /opt/exadata_ovm/exadata.img.domu_maker
A short lookup through the script revealed the nature of the error – apparently we had to add appropriate the bridge to both DOM0s. So we did:
[root@exa2dbadm01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 #### DO NOT REMOVE THESE LINES #### #### %GENERATED BY CELL% #### DEVICE=eth1 BOOTPROTO=none ONBOOT=yes HOTPLUG=no BRIDGE=vmeth1 NM_CONTROLLED=no ETHTOOL_OPTS="-K eth1 lro off" IPV6INIT=no [root@exa2dbadm01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-vmeth1 #### DO NOT REMOVE THESE LINES #### #### %GENERATED BY CELL% #### DEVICE=vmeth1 TYPE=Bridge BOOTPROTO=none ONBOOT=yes DELAY=0 ETHTOOL_OPTS="-K eth1 lro off" IPADDR=10.20.10.60 NETMASK=255.255.0.0 GATEWAY=10.20.0.1 NETWORK=10.20.0.0 BROADCAST=10.20.255.255 [root@exa2dbadm01 ~]# ip a show eth1 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmeth1 state UP qlen 1000 link/ether 00:21:28:a2:aa:ad brd ff:ff:ff:ff:ff:ff inet6 fe80::221:28ff:fea2:aaad/64 scope link valid_lft forever preferred_lft forever [root@exa2dbadm01 ~]# ip a show vmeth1 13: vmeth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 00:21:28:a2:aa:ad brd ff:ff:ff:ff:ff:ff inet 10.20.10.60/16 brd 10.20.255.255 scope global vmeth1
After this operation we got success – other error 😉
[ERROR ] Unable to get map of the virtual network interfaces
Fortunately the script /opt/exadata_ovm/exadata.img.domu_maker is well commented – and here is what we found:
# Get existing bridges from the cell config of "this" dom0 from /opt/oracle.cellos/cell.conf # Check for mismatch between existing bridges in cell.conf and what "brctl show" has
Following that hint we have updated the XML file "/opt/oracle.cellos/cell.conf" and that was it – problem solved 🙂
The missing entry it the file looked like this:
<Interfaces> <Bridge>vmeth1</Bridge> <Gateway>10.20.0.1</Gateway> <Hostname>exa2dbadm01.arrowecs.hub</Hostname> <IP_address>10.20.10.60</IP_address> <IP_enabled>yes</IP_enabled> <IP_ssh_listen>enabled</IP_ssh_listen> <Inet_protocol>IPv4</Inet_protocol> <Name>eth1</Name> <Net_type>Management</Net_type> <Netmask>255.255.0.0</Netmask> <State>1</State> <Status>UP</Status> <Vlan_id>0</Vlan_id> </Interfaces>
Well, that’s it. Hope it saves someone a few hours of searching 😉
Of course you have to remember to edit appropriate files on all DOM0 machines in your Exadata configuration and to be very careful while editing the config files.
Of course the better solution would be to double check your OEDA configuration files before reimaging… But it’s better to alter some configs, than start everything from scratch.