Ask Your Question
0

[fuel] Reset ODL configuration ?

asked 2016-06-02 07:53:09 -0700

dbalsige gravatar image

updated 2016-06-09 07:07:07 -0700

Hi there,

I have successfully installed brahmaputra with fuel on baremetal. 3 ctrl (1odl), 2 compute. I used ODL fuel plugin and configured to manage L3 traffic by ODL.

Cloud was usable, floating IP assignment worked and traffic went in/out of test VM. However since some hours, no connectivity can be made anymore to external networks, neither router external interfaces nor floating IPs assigned to VMs are pingable. I deleted all routers networks and VMs, as well I returned every floating IP to the pool.

I think ODL has messed up with some soft switches on the nodes. On ODL GUI I see 2 disjunct set of switches and I remember having seen they were all connected when the cloud was working. How I can reset ODL to a sane state? Meaning all openflow switches are connected together again? How can I debug ODL managed switches on the nodes? Of course there are no neutron agents and net namespaces on the nodes anymore, since ODL manages all cloud traffic.

Any help appreciated. D


Updated Question:

Thank you for the help. I see some br-ex are not connected to ODL, not only on ctrl or compute nodes, it is mixed. In my understanding all br-ex should be connected to ODL, since N-S floating IP traffic goes directly out of computes br-ex and SNAT traffic out of br-ex of the ctrl node which hosts the active DHCP agent, right?

In a first step I deleted all the OpenStack networks, and routers, neutron net-list, port-list and router-list is empty. Secondly I reconnected all disconneted br-ex by ovs-vsctl set-controller tcp:<odl_ip>:6653 Third I stopped ODL and cleared the data directory mentioned in the answer below. I restarted ODL and all openvswitch-switch services on the node and waited until they show a connected=true for all br-ex and br-int as well as the manager. Indeed now all switches are connected again together in the ODL GUI. However I see still active flows in br-int for my old networks and security rules, which I think is not correct. I think somehow ODL is out of sync with the neutron database. I am not sure what to do now, manually remove flows from br-int ? What is a sane default of flows in both br-int and br-ex when no networks routers or anything else exists? How I can reset the switches and ODL to that state ?

--Update 2: Thank you very much for the reply. I tried all of the proposed methods to get ODL back in a sane state. I was not able to. ODL had some NULL pointer exception when creating the provider network, which I deleted before with neutron. also I deleted flows by hand, but ODL tried to recreate them as soon as bridges were reconnected. So I just reset the fuel environment and re-deployed it again, without changing anything. Deployment went fine. control plane is up. However I was not even able to ping the admin tenants router public interface after ...

(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
1

answered 2016-06-02 09:43:21 -0700

mskalski gravatar image

updated 2016-06-03 04:55:15 -0700

Hi,

It sounds like OVS switches were disconnected from ODL. What is a current status of ovs on controllers and computes? You can use:

ovs-vsctl show

command and check if br-int bridge is created on ovs - it is a ODL responsibility to do that. If yes you can further investigate flows created on this bridge:

ovs-ofctl -O OpenFlow13 dump-flows br-int

and check if there are entries which correspond to your current tenants networks configuration.

It could be helpful to check logs in /var/log/openvswitch/* to see if there is any indication of problems related to ODL ovsdb manager communication. Also I would expect errors presence in /opt/opendaylight/data/log/* files.

To restore env to operational state I would try remove all networks and routers, stop opendaylight service, clean up /opt/opendaylight/data/ directory, start odl and if needed also restart openvswitch services on nodes.

UPDATE

Regarding flows which apply for non-existing network you may try to remove them manually, I believe that on the beginning ODL only initialize tables:

root@node-22:~# ovs-ofctl -O OpenFlow13 dump-flows br-int
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=2337.506s, table=0, n_packets=0, n_bytes=0, dl_type=0x88cc actions=CONTROLLER:65535
cookie=0x0, duration=2337.506s, table=0, n_packets=8, n_bytes=648, priority=0 actions=goto_table:20
cookie=0x0, duration=2337.506s, table=20, n_packets=8, n_bytes=648, priority=0 actions=goto_table:30
cookie=0x0, duration=2337.506s, table=30, n_packets=8, n_bytes=648, priority=0 actions=goto_table:40
cookie=0x0, duration=2337.506s, table=40, n_packets=8, n_bytes=648, priority=0 actions=goto_table:50
cookie=0x0, duration=2337.506s, table=50, n_packets=8, n_bytes=648, priority=0 actions=goto_table:60
cookie=0x0, duration=2337.506s, table=60, n_packets=8, n_bytes=648, priority=0 actions=goto_table:70
cookie=0x0, duration=2337.506s, table=70, n_packets=8, n_bytes=648, priority=0 actions=goto_table:80
cookie=0x0, duration=2337.506s, table=80, n_packets=8, n_bytes=648, priority=0 actions=goto_table:90
cookie=0x0, duration=2337.506s, table=90, n_packets=8, n_bytes=648, priority=0 actions=goto_table:100
cookie=0x0, duration=2337.506s, table=100, n_packets=8, n_bytes=648, priority=0 actions=goto_table:110
cookie=0x0, duration=2337.506s, table=110, n_packets=8, n_bytes=648, priority=0 actions=drop

root@node-22:~# ovs-ofctl -O OpenFlow13 dump-flows br-ex
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=2374.489s, table=0, n_packets=0, n_bytes=0, dl_type=0x88cc actions=CONTROLLER:65535
cookie=0x0, duration=2374.489s, table=0, n_packets=5242, n_bytes=274454, priority=0 actions=NORMAL

You can also wipe out ovsdb database by stopping openvswtich service and remove files from /var/lib/openvswitch and /etc/openvswitch. But this approach will require from you setting manually ODL as an manger for ovs and adding informations about providermappings and localip in case of vxlan segmentation, you can find details here.

Also remember that ODL use lazy initialization and br-ex bridge maybe connected after you add external network.

edit flag offensive delete link more

Comments

Thank you very much. That really helps. On 3 nodes (2 ctrl and one compute) br-ex has no controller connection to ODL, br-int is fine though. manager connection is also fine. Thanks also for the hint with -O OpenFlow13. However I have still some issues (see my updated question)

dbalsige ( 2016-06-02 13:40:59 -0700 )edit
0

answered 2016-06-02 08:26:18 -0700

FernandoBecerra gravatar image

hello, did you have any problems when installing Fuel ?, you kept some tutorial ?. I have many problems when install Fuel.

edit flag offensive delete link more

Comments

Yes I had issues with OPNFV installation tools. Apex was sort of tricky since I am neither a TripleO nor an Ironic expert, so I gave Fuel a try. Apart from the ISO not booting on my H/W it went mostly fine. Some hints:1. The fuel host needs Internet 2. Do the network connectivity check before deploy

dbalsige ( 2016-06-02 14:32:02 -0700 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

[hide preview]

Question Tools

Follow
1 follower

Stats

Asked: 2016-06-02 07:53:09 -0700

Seen: 446 times

Last updated: Jun 09 '16