lkml.org 
[lkml]   [2019]   [Dec]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH RFC 04/15] KVM: Implement ring-based dirty memory tracking
On Tue, Dec 17, 2019 at 01:17:29PM +0800, Tian, Kevin wrote:
> > From: Tian, Kevin
> > Sent: Tuesday, December 17, 2019 10:29 AM
> >
> > > From: Paolo Bonzini
> > > Sent: Monday, December 16, 2019 6:08 PM
> > >
> > > [Alex and Kevin: there are doubts below regarding dirty page tracking
> > > from VFIO and mdev devices, which perhaps you can help with]
> > >
> > > On 15/12/19 18:21, Peter Xu wrote:
> > > > init_rmode_tss
> > > > vmx_set_tss_addr
> > > > kvm_vm_ioctl_set_tss_addr [*]
> > > > init_rmode_identity_map
> > > > vmx_create_vcpu [*]
> > >
> > > These don't matter because their content is not visible to userspace
> > > (the backing storage is mmap-ed by __x86_set_memory_region). In fact, d
> > >
> > > > vmx_write_pml_buffer
> > > > kvm_arch_write_log_dirty [&]
> > > > kvm_write_guest
> > > > kvm_hv_setup_tsc_page
> > > > kvm_guest_time_update [&]
> > > > nested_flush_cached_shadow_vmcs12 [&]
> > > > kvm_write_wall_clock [&]
> > > > kvm_pv_clock_pairing [&]
> > > > kvmgt_rw_gpa [?]
> > >
> > > This then expands (partially) to
> > >
> > > intel_gvt_hypervisor_write_gpa
> > > emulate_csb_update
> > > emulate_execlist_ctx_schedule_out
> > > complete_execlist_workload
> > > complete_current_workload
> > > workload_thread
> > > emulate_execlist_ctx_schedule_in
> > > prepare_execlist_workload
> > > prepare_workload
> > > dispatch_workload
> > > workload_thread
> > >
> > > So KVMGT is always writing to GPAs instead of IOVAs and basically
> > > bypassing a guest IOMMU. So here it would be better if kvmgt was
> > > changed not use kvm_write_guest (also because I'd probably have nacked
> > > that if I had known :)).
> >
> > I agree.
> >
> > >
> > > As far as I know, there is some work on live migration with both VFIO
> > > and mdev, and that probably includes some dirty page tracking API.
> > > kvmgt could switch to that API, or there could be VFIO APIs similar to
> > > kvm_write_guest but taking IOVAs instead of GPAs. Advantage: this would
> > > fix the GPA/IOVA confusion. Disadvantage: userspace would lose the
> > > tracking of writes from mdev devices. Kevin, are these writes used in
> > > any way? Do the calls to intel_gvt_hypervisor_write_gpa covers all
> > > writes from kvmgt vGPUs, or can the hardware write to memory as well
> > > (which would be my guess if I didn't know anything about kvmgt, which I
> > > pretty much don't)?
> >
> > intel_gvt_hypervisor_write_gpa covers all writes due to software mediation.
> >
> > for hardware updates, it needs be mapped in IOMMU through
> > vfio_pin_pages
> > before any DMA happens. The ongoing dirty tracking effort in VFIO will take
> > every pinned page through that API as dirtied.
> >
> > However, currently VFIO doesn't implement any vfio_read/write_guest
> > interface yet. and it doesn't make sense to use vfio_pin_pages for software
> > dirtied pages, as pin is unnecessary and heavy involving iommu invalidation.
>
> One correction. vfio_pin_pages doesn't involve iommu invalidation. I should
> just mean that pinning the page is not necessary. We just need a kvm-like
> interface based on hva to access.
>
And can we propose to differentiate read and write when calling vfio_pin_pages, e.g.
vfio_pin_pages_read, vfio_pin_pages_write? Otherwise, calling to
vfio_pin_pages will unnecessarily cause read pages to be dirty and
sometimes reading guest pages is a way for device model to track dirty
pages.

> >
> > Alex, if you are OK we'll work on such interface and move kvmgt to use it.
> > After it's accepted, we can also mark pages dirty through this new interface
> > in Kirti's dirty page tracking series.
> >

\
 
 \ /
  Last update: 2019-12-17 06:35    [W:0.141 / U:4.508 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site