See how open vswitch compares to other vendors in security performance
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: fix GSO userspace truncation underflow
OVSACTIONATTRTRUNC currently stores a delta from the original skb length in OVSCB(skb)->cutlen. When a later userspace action segments a GSO skb, queuegsopackets() reuses that delta for each smaller segment. A segment can then reach queueuserspacepacket() with cutlen greater than skb->len, underflowing the length passed to skbzerocopy().
Store the maximum preserved length instead and bound each consumer against the current skb length. Use U32MAX as the no-truncation sentinel so the value remains valid if skb geometry changes before a consumer handles it.
A flaw was found in OVN (Open Virtual Network). A remote attacker, by sending crafted DHCPv6 (Dynamic Host Configuration Protocol for IPv6) SOLICIT packets with an inflated Client ID length, could cause the ovn-controller to read beyond the bounds of a packet. This out-of-bounds read can lead to the disclosure of sensitive information stored in heap memory, which is then returned to the attacker's virtual machine port.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: use RCU protection in ovsvportcmdfillinfo()
ovsvportcmdfillinfo() can be called without RTNL or RCU.
Use RCU protection and devnetrcu() to avoid potential UAF.
In the Linux kernel, the following vulnerability has been resolved:
Revert "openvswitch: switch to per-action label counting in conntrack"
Currently, ovsctsetlabels() is only called for confirmed conntrack entries (ct) within ovsctcommit(). However, if the conntrack entry does not have the labelsext extension, attempting to allocate it in ovsctgetconnlabels() for a confirmed entry triggers a warning in nfctextadd():
WARNON(nfctisconfirmed(ct));
This happens when the conntrack entry is created externally before OVS increments net->ct.labelsused. The issue has become more likely since commit fcb1aa5163b1 ("openvswitch: switch to per-action label counting in conntrack"), which changed to use per-action label counting and increment net->ct.labelsused when a flow with ct action is added.
Since there’s no straightforward way to fully resolve this issue at the moment, this reverts the commit to avoid breaking existing use cases.
In the Linux kernel, the following vulnerability has been resolved:
net: openvswitch: fix nested key length validation in the set() action
It's not safe to access nlalen(ovskey) if the data is smaller than the netlink header. Check that the attribute is OK first.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: Fix unsafe attribute parsing in outputuserspace()
This patch replaces the manual Netlink attribute iteration in outputuserspace() with nlaforeachnested(), which ensures that only well-formed attributes are processed.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: defer tunnel netdevput to RCU release
ovsnetdevtunneldestroy() may run after NETDEVUNREGISTER already detached the device. Dropping the netdev reference in destroy can race with concurrent readers that still observe vport->dev.
Do not release vport->dev in ovsnetdevtunneldestroy(). Instead, let vportnetdevfree() drop the reference from the RCU callback, matching the non-tunnel destroy path and avoiding additional synchronization under RTNL.
In the Linux kernel, the following vulnerability has been resolved:
net: openvswitch: fix potential UAF on meter attach failure
While attaching a newly created meter attachmeter() function makes the new meter visible to other CPUs but can still fail afterwards. On failure, it detaches the meter back and returns an error.
However, this is an unexpected behavior for the ovsmetercmdset() that uses a plain kfree(meter) on attach failure without waiting for RCU readers to stop using it, assuming it was never visible.
This is never a problem for ovs-vswitchd as it always creates meters before creating any flows that use them. But the UAF can be triggered with a custom application using uAPI:
BUG: KASAN: slab-use-after-free in ovsmeterexecute (net/openvswitch/meter.c:653) Read of size 8 at addr ffff88810d152650 by task meter/2508
Call Trace: ovsmeterexecute (net/openvswitch/meter.c:653) doexecuteactions (net/openvswitch/actions.c:1407) ovsexecuteactions (net/openvswitch/actions.c:1584) ovspacketcmdexecute (net/openvswitch/datapath.c:703) ... netlinksendmsg (afnetlink.c:1900)
Allocated by task 2519: kasankmalloc (mm/kasan/common.c:398 mm/kasan/common.c:415) ovsmetercmdset (net/openvswitch/meter.c:422) ... netlinksendmsg (afnetlink.c:1900)
Freed by task 2519: kfree (mm/slub.c:2705 mm/slub.c:6405 mm/slub.c:6720) ovsmetercmdset (net/openvswitch/meter.c:479) ... netlinksendmsg (afnetlink.c:1900)
Fix that by making sure attachmeter() doesn't make the meter visible until all the checks are done and the function can't fail anymore.
This also makes sure the "hash" value is calculated after the potential re-sizing of the table.
Reported by Trend Micro's Zero Day Initiative as ZDI-CAN-31642.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: validate MPLS set/setmasked payload length
validateset() accepted OVSKEYATTRMPLS as variable-sized payload for SET/SETMASKED actions. In action handling, OVS expects fixed-size MPLS key data (struct ovskeympls).
Use the already normalized keylen (masked case included) and reject non-matching MPLS action key sizes.
Reject invalid MPLS action payload lengths early.
Multiple versions of OVN (Open Virtual Network) are vulnerable to crafted DHCPv6 packets that could potentially read out-of-bounds, leaking adjacent info stored on the heap.
OVN supports configuring DHCPv6 options for Logical Switch Ports. When configured we allow handling of DHCPv6 requests in a userspace thread called pinctrl. The thread accesses user-controlled packet data and copies some of it in the process of creating a reply packet.
When building a DHCPv6 ADVERTISE reply, the handler echoes the Client ID option using the option's self-declared length without validating it against the actual packet bounds. A workload can send a crafted DHCPv6 SOLICIT with an inflated Client ID length field, causing ovn-controller to copy heap memory beyond the valid packet data into the reply. The reply is then delivered back to the attacker's VM port.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: cap upcall PID array size and pre-size vport replies
The vport netlink reply helpers allocate a fixed-size skb with nlmsgnew(NLMSGDEFAULTSIZE, ...) but serialize the full upcall PID array via ovsvportgetupcallportids(). Since ovsvportsetupcallportids() accepts any non-zero multiple of sizeof(u32) with no upper bound, a CAPNETADMIN user can install a PID array large enough to overflow the reply buffer, causing nlaput() to fail with -EMSGSIZE and hitting BUGON(err < 0). On systems with unprivileged user namespaces enabled (e.g., Ubuntu default), this is reachable via unshare -Urn since OVS vport mutation operations use GENLUNSADMINPERM.
kernel BUG at net/openvswitch/datapath.c:2414! Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI CPU: 1 UID: 0 PID: 65 Comm: poc Not tainted 7.0.0-rc7-00195-geb216e422044 #1 RIP: 0010:ovsvportcmdset+0x34c/0x400 Call Trace: <TASK> genlfamilyrcvmsgdoit (net/netlink/genetlink.c:1116) genlrcvmsg (net/netlink/genetlink.c:1194) netlinkrcvskb (net/netlink/afnetlink.c:2550) genlrcv (net/netlink/genetlink.c:1219) netlinkunicast (net/netlink/afnetlink.c:1344) netlinksendmsg (net/netlink/afnetlink.c:1894) syssendto (net/socket.c:2206) x64syssendto (net/socket.c:2209) dosyscall64 (arch/x86/entry/syscall64.c:63) entrySYSCALL64afterhwframe (arch/x86/entry/entry64.S:130) </TASK> Kernel panic - not syncing: Fatal exception
Reject attempts to set more PIDs than nrcpuids in ovsvportsetupcallportids(), and pre-compute the worst-case reply size in ovsvportcmdmsgsize() based on that bound, similar to the existing ovsdpcmdmsgsize(). nrcpuids matches the cap already used by the per-CPU dispatch configuration on the datapath side (ovsdpcmdfillinfo() serialises at most nrcpuids PIDs), so the two sides stay consistent.
When generating an ICMP Destination Unreachable or Packet Too Big response, the handler copies a portion of the original packet into the ICMP error body using the IP header's self-declared total length (iptotlen for IPv4, ip6plen for IPv6) without validating it against the actual packet buffer size. A VM can send a short packet with an inflated IP length field that triggers an ICMP error (e.g., by hitting a reject ACL), causing ovn-controller to read heap memory beyond the valid packet data and include it in the ICMP response sent back to the VM.
A missing upper-bound check in the udpifsetthreads() function of Open vSwitch v3.6.90 allows an attacker with OVSDB write access to request an excessive number of handler or revalidation threads. This can cause a denial of service (DoS) via resource exhaustion.
A flaw was found in Open vSwitch. When Open vSwitch is configured with a conntrack flow using FTP helpers over the userspace datapath, a remote attacker can send a specially crafted FTP stream with an EPASV command exceeding 255 characters. This heap access error can lead to a crash, resulting in a Denial of Service (DoS) for the affected system.
In the Linux kernel, the following vulnerability has been resolved:
net: openvswitch: fix leak of nested actions
While parsing user-provided actions, openvswitch module may dynamically allocate memory and store pointers in the internal copy of the actions. So this memory has to be freed while destroying the actions.
Currently there are only two such actions: ct() and set(). However, there are many actions that can hold nested lists of actions and ovsnlafreeflowactions() just jumps over them leaking the memory.
For example, removal of the flow with the following actions will lead to a leak of the memory allocated by nfcttmplalloc():
actions:clone(ct(commit),0)
Non-freed set() action may also leak the 'dst' structure for the tunnel info including device references.
Under certain conditions with a high rate of flow rotation that may cause significant memory leak problem (2MB per second in reporter's case). The problem is also hard to mitigate, because the user doesn't have direct control over the datapath flows generated by OVS.
Fix that by iterating over all the nested actions and freeing everything that needs to be freed recursively.
New build time assertion should protect us from this problem if new actions will be added in the future.
Unfortunately, openvswitch module doesn't use NLAFNESTED, so all attributes has to be explicitly checked. sample() and clone() actions are mixing extra attributes into the user-provided action list. That prevents some code generalization too.
In the Linux kernel, the following vulnerability has been resolved:
net: openvswitch: fix flow memory leak in ovsflowcmdnew
Syzkaller reports a memory leak of newflow in ovsflowcmdnew() as it is not freed when an allocation of a key fails.
BUG: memory leak unreferenced object 0xffff888116668000 (size 632): comm "syz-executor231", pid 1090, jiffies 4294844701 (age 18.871s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000defa3494>] kmemcachezalloc include/linux/slab.h:654 [inline] [<00000000defa3494>] ovsflowalloc+0x19/0x180 net/openvswitch/flowtable.c:77 [<00000000c67d8873>] ovsflowcmdnew+0x1de/0xd40 net/openvswitch/datapath.c:957 [<0000000010a539a8>] genlfamilyrcvmsgdoit+0x22d/0x330 net/netlink/genetlink.c:739 [<00000000dff3302d>] genlfamilyrcvmsg net/netlink/genetlink.c:783 [inline] [<00000000dff3302d>] genlrcvmsg+0x328/0x590 net/netlink/genetlink.c:800 [<000000000286dd87>] netlinkrcvskb+0x153/0x430 net/netlink/afnetlink.c:2515 [<0000000061fed410>] genlrcv+0x24/0x40 net/netlink/genetlink.c:811 [<000000009dc0f111>] netlinkunicastkernel net/netlink/afnetlink.c:1313 [inline] [<000000009dc0f111>] netlinkunicast+0x545/0x7f0 net/netlink/afnetlink.c:1339 [<000000004a5ee816>] netlinksendmsg+0x8e7/0xde0 net/netlink/afnetlink.c:1934 [<00000000482b476f>] socksendmsgnosec net/socket.c:651 [inline] [<00000000482b476f>] socksendmsg+0x152/0x190 net/socket.c:671 [<00000000698574ba>] syssendmsg+0x70a/0x870 net/socket.c:2356 [<00000000d28d9e11>] syssendmsg+0xf3/0x170 net/socket.c:2410 [<0000000083ba9120>] syssendmsg+0xe5/0x1b0 net/socket.c:2439 [<00000000c00628f8>] dosyscall64+0x30/0x40 arch/x86/entry/common.c:46 [<000000004abfdcf4>] entrySYSCALL64afterhwframe+0x61/0xc6
To fix this the patch rearranges the goto labels to reflect the order of object allocations and adds appropriate goto statements on the error paths.
Found by Linux Verification Center (linuxtesting.org) with Syzkaller.
In the Linux kernel, the following vulnerability has been resolved:
openvswitch: fix memory leak at failed datapath creation
ovsdpcmdnew()->ovsdpchange()->ovsdpsetupcallportids() allocates array via kmalloc. If for some reason newvport() fails during ovsdpcmdnew() dp->upcallportids must be freed. Add missing kfree.
Kmemleak example: unreferenced object 0xffff88800c382500 (size 64): comm "dumpstate", pid 323, jiffies 4294955418 (age 104.347s) hex dump (first 32 bytes): 5e c2 79 e4 1f 7a 38 c7 09 21 38 0c 80 88 ff ff ^.y..z8..!8..... 03 00 00 00 0a 00 00 00 14 00 00 00 28 00 00 00 ............(... backtrace: [<0000000071bebc9f>] ovsdpsetupcallportids+0x38/0xa0 [<000000000187d8bd>] ovsdpchange+0x63/0xe0 [<000000002397e446>] ovsdpcmdnew+0x1f0/0x380 [<00000000aa06f36e>] genlfamilyrcvmsgdoit+0xea/0x150 [<000000008f583bc4>] genlrcvmsg+0xdc/0x1e0 [<00000000fa10e377>] netlinkrcvskb+0x50/0x100 [<000000004959cece>] genlrcv+0x24/0x40 [<000000004699ac7f>] netlinkunicast+0x23e/0x360 [<00000000c153573e>] netlinksendmsg+0x24e/0x4b0 [<000000006f4aa380>] socksendmsg+0x62/0x70 [<00000000d0068654>] syssendmsg+0x230/0x270 [<0000000012dacf7d>] syssendmsg+0x88/0xd0 [<0000000011776020>] syssendmsg+0x59/0xa0 [<000000002e8f2dc1>] dosyscall64+0x3b/0x90 [<000000003243e7cb>] entrySYSCALL64afterhwframe+0x63/0xcd
When generating an ICMP Destination Unreachable or Packet Too Big response, the handler copies a portion of the original packet into the ICMP error body using the IP header's self-declared total length (iptotlen for IPv4, ip6plen for IPv6) without validating it against the actual packet buffer size. A VM can send a short packet with an inflated IP length field that triggers an ICMP error (e.g., by hitting a reject ACL), causing ovn-controller to read heap memory beyond the valid packet data and include it in the ICMP response sent back to the VM.
A flaw was found in Open vSwitch. When Open vSwitch is configured with a conntrack flow using FTP helpers over the userspace datapath, a remote attacker can send a specially crafted FTP stream with an EPASV command exceeding 255 characters. This heap access error can lead to a crash, resulting in a Denial of Service (DoS) for the affected system.
An issue was discovered in Open vSwitch (OvS) 2.7.x through 2.7.6. The decodebundle function inside lib/ofp-actions.c is affected by a buffer over-read issue during BUNDLE action decoding.
Upstream Patch:
https://nvd.nist.gov/vuln/detail/CVE-2018-17206
An issue was discovered in Open vSwitch (OvS) 2.7.x through 2.7.6, affecting ofprotoruleinsert in ofproto/ofproto.c. During bundle commit, flows that are added in a bundle are applied to ofproto in order. If a flow cannot be added (e.g., the flow action is a go-to for a group id that does not exist), OvS tries to revert back all previous flows that were successfully applied from the same bundle. This is possible since OvS maintains list of old flows that were replaced by flows from the bundle. While reinserting old flows, OvS has an assertion failure due to a check on rule state != RULEINITIALIZED. This would work for new flows, but for an old flow the rule state is RULEREMOVED. The assertion failure causes an OvS crash.
Upstream Patch:
https://github.com/openvswitch/ovs/commit/0befd1f3745055c32940f5faf9559be6a14395e6
An issue was discovered in Open vSwitch (OvS) 2.7.x through 2.7.6, affecting parsegrouppropntrselectionmethod in lib/ofp-util.c. When decoding a group mod, it validates the group type and command after the whole group mod has been decoded. The OF1.5 decoder, however, tries to use the type and command earlier, when it might still be invalid. This causes an assertion failure (via OVSNOTREACHED). ovs-vswitchd does not enable support for OpenFlow 1.5 by default.
Upstream Patch:
https://github.com/openvswitch/ovs/commit/4af6da3b275b764b1afe194df6499b33d2bf4cde
In Open vSwitch there is a buffer over-read while parsing the group mod OpenFlow message sent from the controller in lib/ofp-util.c in the function ofputilpullofp15groupmod.
References:
https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332965.html
In Open vSwitch while parsing an OpenFlow role status message, there is a call to the abort() function for undefined role status reasons in the function ofpprintrolestatusmessage in lib/ofp-print.c that may be leveraged toward a remote DoS attack by a malicious switch.
References:
https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332966.html
A vulnerability in openvswitch was found. While parsing an OFPTQUEUEGETCONFIGREPLY type OFP 1.0 message, there is a buffer over-read that is caused by an unsigned integer underflow in the function ofputilpullqueuegetconfigreply10 in lib/ofp-util.c.
References:
https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332711.html
In lib/conntrack.c in the firewall implementation in Open vSwitch, there is a buffer over-read while parsing malformed TCP, UDP, and IPv6 packets in the functions extractl3ipv6, extractl4tcp, and extractl4udp that can be triggered remotely.
References:
https://mail.openvswitch.org/pipermail/ovs-dev/2017-March/329323.html
Andreas Beckmann debian reports:
openvswitch-pki creates the following world writable directories during installation:
drwx-wx-wx 2 root root 40 Aug 1 05:32 /var/lib/openvswitch/pki/controllerca/incoming drwx-wx-wx 2 root root 40 Aug 1 05:32 /var/lib/openvswitch/pki/switchca/incoming
Even if an ordinary local user cannot list the contents of the directory, he may correctly derive/guess filenames (unless they are exclusively $(mktemp)) and delete and replace files in there.
I don't know how openvswitch-pki works, how it uses this directory, what probelms could possibly arise out of this.
References: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683665
Please note on Fedora 16 and 17 run the command:
/usr/bin/ovs-pki --force init
to create the directories.
Description ===========
Multiple versions of Open vSwitch are vulnerable to crafted Geneve packets causing invalid memory accesses and potential denial of service. Triggering the vulnerability requires that Open vSwitch has flow hardware offload with Linux TC flower enabled (otherconfig:hw-offload=true). It is not enabled by default.
The issue is caused by insufficient validation of Geneve metadata fields in the offload path. Open vSwitch versions 2.12 and newer are affected.
The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned CVE-2023-3966 identifier to this issue.
Mitigation ==========
For any version of Open vSwitch, disabling flow hardware offload will prevent the issue (setting otherconfig:hw-offload=false and reboot the system). We do not recommend attempting to mitigate the vulnerability this way because it may impact the overall system throughput.
By default, flow hardware offload support is not enabled.
Fix ===
Patches to fix this vulnerability in Open vSwitch 2.17 and newer:
3.2.x: https://github.com/openvswitch/ovs/commit/2cfbcd5247ed0fd941c1ebb9f4adb952b67fe13a 3.1.x: https://github.com/openvswitch/ovs/commit/91e621bd5abab19954bec09c7d27c59acdf607b1 3.0.x: https://github.com/openvswitch/ovs/commit/935cd1d574c6f432a451df8941374ffb36d767d9 2.17.x: https://github.com/openvswitch/ovs/commit/b8657dada9641fbd2bd3a3f882e0862448d60910
Original patch is located at: https://mail.openvswitch.org/pipermail/ovs-dev/2024-February/411702.html
Recommendation ==============
We recommend that users of Open vSwitch apply the linked patches, or upgrade to a known patched version of Open vSwitch. These include:
v3.2.2 v3.1.4 v3.0.6 v2.17.9
Acknowledgments ===============
The Open vSwitch team wishes to thank the reporters:
Timothy Redaelli <tredaelli () redhat com> Haresh Khandelwal <hakhande () redhat com>
Description ===========
In multiple versions of Open vSwitch, if OpenFlow rules on a switch contain a match on a Target Address (ndtarget) of Neighbor Discovery IPv6 packets (Neighbor Solicitation or Neighbor Advertisement) without also matching on ICMPv6 Code (icmpcode or icmpv6code) field being zero, the match on the Target Address can be ignored and the specified actions may be executed for a packet with a different Target Address.
This constitutes vulnerability if such OpenFlow rules are used in order to provide Neighbor Discovery anti-spoofing protection. For example, the following set of rules may allow packets with any ndtarget, even though it should only allow packets with the 2001::1 Target:
priority=10 icmp6,icmpv6type=136,ndtarget=2001::1 actions=<allow> priority=0 icmp6 actions=drop
The issue is caused by the difference between the OpenFlow specification that only lists ICMPV6 TYPE=135 or ICMPV6 TYPE=136 as a prerequisite for the IPV6NDTARGET and datapath implementations that treat ICMPV6CODE=0 as a requirement for a packet to have the Target Address option. This leads to creation of an overly broad datapath flow that matches packets regardless of the Target Address value.
Triggering the issue depends on the order in which packets are seen by the switch.
Open vSwitch versions 2.1 and newer are affected.
The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned CVE-2023-5366 identifier to this issue.
Mitigation ==========
For any affected version of Open vSwitch, adding an icmpv6code=0 match to an OpenFlow rule will mitigate the issue. For example:
priority=10 icmp6,icmpv6type=136,icmpv6code=0,ndtarget=2001::1 \ actions=<allow> priority=0 icmp6 actions=drop
'icmpv6code' corresponds to OXMOFICMPV6CODE staring with OF1.2 or NXMOFICMPV6CODE Nicira Extension before that.
Fix ===
Patches to fix this vulnerability in Open vSwitch 2.17 and newer has been applied to the appropriate branches:
3.2.x: https://github.com/openvswitch/ovs/commit/7570744c5add3a91b468c4ffa5bc73ef1f5bb18a 3.1.x: https://github.com/openvswitch/ovs/commit/d3f9eab1abbb6a11c2a166472c184f54fd740bf1 3.0.x: https://github.com/openvswitch/ovs/commit/a6c0a3deb268f34faef0062e2c05ece563d50ecb 2.17.x: https://github.com/openvswitch/ovs/commit/e235a421fbdb0c70176e8a3bef13bf7e2056cbc1
The original patch is located at: https://mail.openvswitch.org/pipermail/ovs-dev/2024-February/411701.html
Note for those performing selective patch backports:
The following change is a prerequisite for the fix and must also be applied for the fix to work correctly: https://github.com/openvswitch/ovs/commit/489553b1c21692063931a9f50b6849b23128443c
Recommendation ==============
We recommend that users of Open vSwitch apply the respective patches, or upgrade to a known patched version of Open vSwitch. These include:
3.2.2 3.1.4 3.0.6 2.17.9
However, while patches are enough to fix the issue, checking both the ICMPv6 Code and Type is the correct way to validate Neighbor Discovery packets according to RFC 4861. With this in mind, we recommend that users follow the mitigation strategy even with the patches applied, i.e. add an extra match criteria on 'icmpv6code' field.
Acknowledgments ===============
The Open vSwitch team wishes to thank the reporters:
Alex Katz <akatz () redhat com> Slawomir Kaplonski <skaplons () redhat com>
Description ===========
Mitigation ==========
Fix ===
The original patch is located at: https://mail.openvswitch.org/pipermail/ovs-dev/2025-January/419993.html
Recommendation ============== v22.03.8 v24.03.5 v24.09.2
Acknowledgments ===============
The OVN team wishes to thank the reporters:
Marius Berntsberg - marius () redpill-linpro com Trygve Vea - tv () redpill-linpro com Tore Anderson - tore () redpill-linpro com Rodolfo Alonso - ralonsoh () redhat com Jay Faulkner (Openstack VMT) - jay () jvf cc Brian Haley (Openstack/Neutron) - haleyb.dev () gmail com