Date
1 - 3 of 3
AWS Enhanced Networking
|
Danny Berger <dpb587@...>
Hi all - I'm interested in the "Enhanced Networking" capabilities on the
BOSH-published AWS HVM light stemcells, with particular interest in the lower latencies it can bring. Amazon has two stemcell-relevant requirements <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html#enhanced-networking-requirements>, which are not entirely being met (based on the bosh-aws-xen-hvm-ubuntu-trusty-go_agent/2992 VM I was looking at): * Kernel Module ixgbevf, recommended 2.14.2+ (stemcell currently has 2.12.1-k) * Image Attribute sriovNetSupport (not currently configured) In terms of changes, I think the first and easiest change would be to start registering the published AMIs with the additional `--sriov-net-support simple` option, a la AWS CLI. That would at least enable the enhanced networking capabilities, with the second possible change being to upgrade lixgbevf for the latest bug fixes and minor improvements. In terms of trying to make this change, I'm not sure what the next step is. Can anyone offer further guidance on this? Or advise me on how I can build the stemcells for internal use (which I can then register as sriov-capable)? Or is there a better way to go about this? Thanks for your time! Danny -- Danny Berger http://dpb587.me |
|
|
|
Dmitriy Kalinin
Thanks for looking into this!
toggle quoted message
Show quoted text
I would be very interested in enabling enhanced networking as a default for AWS stemcells. It sounds like you already identified needed changes to make that happen. Here is some more details around the code: 1) modify stemcell builder to install correct version of ixgbevf - add sriov_net_support => true to https://github.com/cloudfoundry/bosh/blob/master/bosh-stemcell/lib/bosh/stemcell/infrastructure.rb#L96 to signify that that stemcell supports it - add a new stage to https://github.com/cloudfoundry/bosh/tree/master/stemcell_builder/stages with vendored source for ixgbevf - include stage name in a list of stages to run when building stemcells: https://github.com/cloudfoundry/bosh/blob/master/bosh-stemcell/lib/bosh/stemcell/stage_collection.rb - make sure it happens after system_kernel stage 1.1) see https://github.com/cloudfoundry/bosh/blob/master/bosh-stemcell how to build your own stemcell 2) modify aws cpi to set sriovNetSupport attributes when creating stemcells - include the attribute in https://github.com/cloudfoundry/bosh/blob/master/bosh_aws_cpi/lib/cloud/aws/stemcell_creator.rb#L84 if stemcell's cloud_properties includes sriov_net_support => true - this code path is used by our CI to create light stemcells but also when full AWS stemcell is uploaded to the director, so it covers all cases - make sure that stemcells still boot on instance types that do not support sriov even if stemcell supports it - make sure that stemcell booting on instance types that support sriov actually take advantage of that feature 3) submit PR and we will get that into our pipelines! On Mon, Jun 22, 2015 at 1:06 PM, Danny Berger <dpb587(a)gmail.com> wrote:
Hi all - I'm interested in the "Enhanced Networking" capabilities on the |
|
|
|
James Bayer
thanks for doing the research danny!
please let us know if you try working on this. we'll definitely try and help if you get stuck. On Mon, Jun 22, 2015 at 5:04 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote: Thanks for looking into this! -- Thank you, James Bayer |
|
|