bhyve VM stopped to boot after moving virtio disks

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

bhyve VM stopped to boot after moving virtio disks

Wojciech Puchar-8
i have bhyve VM  (linux) with 2 virtual disks.

they were backed on geli encrypted partitions on SSD

as it is rarely used VM i wanted to move it to HDD file.

i just did

dd if=/dev/partition1.eli of=file1 bs=1m
dd if=/dev/partition2.eli of=file2 bs=1m

there were no errors when doing dd

and changed /dev/partition1.eli to file1 and same with second in
virtio-blk filename.

Everything else is unchanged.

now it doesn't boot. on UEFI virtual framebuffer i see

Boot Failed. EFI Misc Device
Boot Failed. EFI Misc Device 1
.



Could you give me any clues?
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Ian Lepore-3
On Fri, 2019-04-19 at 23:00 +0200, Wojciech Puchar wrote:

> i have bhyve VM  (linux) with 2 virtual disks.
>
> they were backed on geli encrypted partitions on SSD
>
> as it is rarely used VM i wanted to move it to HDD file.
>
> i just did
>
> dd if=/dev/partition1.eli of=file1 bs=1m
> dd if=/dev/partition2.eli of=file2 bs=1m
>
> there were no errors when doing dd
>
> and changed /dev/partition1.eli to file1 and same with second in
> virtio-blk filename.
>
> Everything else is unchanged.
>
> now it doesn't boot. on UEFI virtual framebuffer i see
>
> Boot Failed. EFI Misc Device
> Boot Failed. EFI Misc Device 1
> .
>
>
>
> Could you give me any clues?
>

Are you absolutely certain those encrypted partition sizes are an exact
multiple of 1m?  If not, you should add a 'conv=sync' option to the dd.

-- Ian

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Wojciech Puchar-8
>> Boot Failed. EFI Misc Device
>> Boot Failed. EFI Misc Device 1
>> .
>>
>>
>>
>> Could you give me any clues?
>>
>
> Are you absolutely certain those encrypted partition sizes are an exact
> multiple of 1m?  If not, you should add a 'conv=sync' option to the dd.
no it wasn't. but dd doesn't truncate last block. checked - it got copied
till the end - i see second copy of GPT partition table that was present
in VM.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Ian Lepore-3
On Sat, 2019-04-20 at 17:30 +0200, Wojciech Puchar wrote:

> > > Boot Failed. EFI Misc Device
> > > Boot Failed. EFI Misc Device 1
> > > .
> > >
> > >
> > >
> > > Could you give me any clues?
> > >
> >
> > Are you absolutely certain those encrypted partition sizes are an
> > exact
> > multiple of 1m?  If not, you should add a 'conv=sync' option to the
> > dd.
>
> no it wasn't. but dd doesn't truncate last block. checked - it got
> copied
> till the end - i see second copy of GPT partition table that was
> present
> in VM.
>

dd absolutely will fail to copy the last block of the source if it
isn't exactly the blocksize and you didn't specify conv=sync, and it
will return a zero status when doing so.  It appears you've convinced
yourself otherwise, but for anyone else reading this thread, be aware:
conv=sync is required to copy the last part of the source if it's
smaller than the blocksize.

-- Ian

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Igor Mozolevsky-2
On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote:

<snip>
> dd absolutely will fail to copy the last block of the source if it
> isn't exactly the blocksize and you didn't specify conv=sync, and it
> will return a zero status when doing so.  It appears you've convinced
> yourself otherwise, but for anyone else reading this thread, be aware:
> conv=sync is required to copy the last part of the source if it's
> smaller than the blocksize.


Isn't that contrary to the POSIX Spec? Reading the manual [1], it is
implied (from STDERR part of the main section) that dd will read and
write partial blocks, cf. truncated blocks?..

1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html


--
Igor M.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Ian Lepore-3
On Sat, 2019-04-20 at 18:45 +0100, Igor Mozolevsky wrote:

> On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote:
>
> <snip>
> > dd absolutely will fail to copy the last block of the source if it
> > isn't exactly the blocksize and you didn't specify conv=sync, and
> > it
> > will return a zero status when doing so.  It appears you've
> > convinced
> > yourself otherwise, but for anyone else reading this thread, be
> > aware:
> > conv=sync is required to copy the last part of the source if it's
> > smaller than the blocksize.
>
>
> Isn't that contrary to the POSIX Spec? Reading the manual [1], it is
> implied (from STDERR part of the main section) that dd will read and
> write partial blocks, cf. truncated blocks?..
>
> 1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
>
>

Damn, why do people trim away the entire useful context of an email
thread when replying?  I almost don't want to reply to this because at
this point all the useful info is gone.

In any case, in retrospect, what I said was wrong.  Using conv=sync is
important when the output is a device that has fixed block sizes where
you cannot write a partial block.  When the destination is a file where
short writes work correctly, it will copy the entire source correctly.

-- Ian

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

freebsd-hackers mailing list
In reply to this post by Igor Mozolevsky-2


On 2019-Apr-20, at 10:45, Igor Mozolevsky <igor at hybrid-lab.co.uk> wrote:

> On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote:
>
> <snip>
>> dd absolutely will fail to copy the last block of the source if it
>> isn't exactly the blocksize and you didn't specify conv=sync, and it
>> will return a zero status when doing so.  It appears you've convinced
>> yourself otherwise, but for anyone else reading this thread, be aware:
>> conv=sync is required to copy the last part of the source if it's
>> smaller than the blocksize.
>
>
> Isn't that contrary to the POSIX Spec? Reading the manual [1], it is
> implied (from STDERR part of the main section) that dd will read and
> write partial blocks, cf. truncated blocks?..
>
> 1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

Have you tried as Ian suggests to see if it made a difference?
Ian seems to be speaking FreeBSD, not necessarily POSIX. (I'm
talking actual FreeBSD behavior here, not necessarily documented
behavior.)

It turns out that https://www.freebsd.org/cgi/man.cgi?dd(1) does say:

     Normally, data resulting from input or conversion or both are aggregated
     into output blocks of the specified size. After the end of input is
     reached, any remaining output is written as a block.  This means that the
     final output block may be shorter than the output block size.

where conv=sync is described as:

sync
     Pad every input block to the input buffer size. Spaces
     are used for pad bytes if a block oriented conversion
     value is specified, otherwise NUL bytes are used.

(So sync changes the overall input size when it adds pad bytes, at
least as documented.)

It appears to me that Ian's expectations of the actual behavior and
the FreeBSD documentation of such are not a match. Absent testing,
I'd bet on Ian, even if it means FreeBSD dd has a bug.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Rodney W. Grimes-6
>
>
> On 2019-Apr-20, at 10:45, Igor Mozolevsky <igor at hybrid-lab.co.uk> wrote:
>
> > On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote:
> >
> > <snip>
> >> dd absolutely will fail to copy the last block of the source if it
> >> isn't exactly the blocksize and you didn't specify conv=sync, and it
> >> will return a zero status when doing so.  It appears you've convinced
> >> yourself otherwise, but for anyone else reading this thread, be aware:
> >> conv=sync is required to copy the last part of the source if it's
> >> smaller than the blocksize.
> >
> >
> > Isn't that contrary to the POSIX Spec? Reading the manual [1], it is
> > implied (from STDERR part of the main section) that dd will read and
> > write partial blocks, cf. truncated blocks?..
> >
> > 1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
>
> Have you tried as Ian suggests to see if it made a difference?
> Ian seems to be speaking FreeBSD, not necessarily POSIX. (I'm
> talking actual FreeBSD behavior here, not necessarily documented
> behavior.)
>
> It turns out that https://www.freebsd.org/cgi/man.cgi?dd(1) does say:
>
>      Normally, data resulting from input or conversion or both are aggregated
>      into output blocks of the specified size. After the end of input is
>      reached, any remaining output is written as a block.  This means that the
>      final output block may be shorter than the output block size.
>
> where conv=sync is described as:
>
> sync
>      Pad every input block to the input buffer size. Spaces
>      are used for pad bytes if a block oriented conversion
>      value is specified, otherwise NUL bytes are used.
>
> (So sync changes the overall input size when it adds pad bytes, at
> least as documented.)
>
> It appears to me that Ian's expectations of the actual behavior and
> the FreeBSD documentation of such are not a match. Absent testing,
> I'd bet on Ian, even if it means FreeBSD dd has a bug.

So I tested:
root@x230a:/tmp/ddtest # dd if=short of=long bs=1m
0+1 records in
0+1 records out
1024 bytes transferred in 0.000101 secs (10171444 bytes/sec)
root@x230a:/tmp/ddtest # ls -lag
total 62
drwxr-xr-x   2 root  wheel     4 Apr 20 19:16 .
drwxrwxrwt  44 root  wheel   283 Apr 20 19:16 ..
-rw-r--r--   1 root  wheel  1024 Apr 20 19:16 long
-rw-r--r--   1 root  wheel  1024 Apr 20 19:16 short

And on files it is as expected, however the issue Ian describes
does infact occur when dealing with block devices that can not
do the "truncated" size block operation, which is extremly rare
as the block devices are typically either 512, 2048 or 4096 and
almost everyone using dd uses block sizes that are large multiples
of this so you never end up hitting that edge of trying to
write a 1k record to a 4k device.  The truncated input record
size is garanteed to be a mutliple of the device block size,
the one case I can think of that would hit this is dd'ing a
512B disk to a 4096 native disk that happened to NOT be of a
size that is a multiple of 4096, something probably VERY rare.

In the case of going from a device to a file the chances
of lost data are nill (and I have been doing that for 30
years and never hit it.)  You DO have to be carefull if your
doing forinsics, conv=sync infact can corrupt your forinsics
copy in that it is the wrong size and then when someone else
does the analysis they get a different results do to this
size change.  Part of this process usually involves verifying
that the size of your image exactly matches the reported
size of the device.

--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Matthias Apitz-4
In reply to this post by Ian Lepore-3
El día sábado, abril 20, 2019 a las 10:56:45a. m. -0600, Ian Lepore escribió:

> > no it wasn't. but dd doesn't truncate last block. checked - it got
> > copied
> > till the end - i see second copy of GPT partition table that was
> > present
> > in VM.
> >
>
> dd absolutely will fail to copy the last block of the source if it
> isn't exactly the blocksize and you didn't specify conv=sync, and it
> will return a zero status when doing so.  It appears you've convinced
> yourself otherwise, but for anyone else reading this thread, be aware:
> conv=sync is required to copy the last part of the source if it's
> smaller than the blocksize.

Writing to a plain file or char device, will write the last block in the
available size of bytes:

$ truss dd if=backupKDE-20190419.tgz of=/dev/null bs=8m
...
read(3,"\M^_\M^_\M^Wi\M-F\^?\n\M-J\M-%T"...,8388608) = 8388608 (0x800000)
write(4,"\M^_\M^_\M^Wi\M-F\^?\n\M-J\M-%T"...,8388608) = 8388608 (0x800000)
read(3,"%\M-b\M^Y!\M^Ud\M^B\\\M^BA \^^"...,8388608) = 8169189 (0x7ca6e5)
                                                      ^^^^^^^
read(3,0x8015cb065,8388608) = 0 (0x0)
write(4,"%\M-b\M^Y!\M^Ud\M^B\\\M^BA \^^"...,8169189) = 8169189 (0x7ca6e5)
                                                       ^^^^^^^
close(4) = 0 (0x0)
clock_getres(0x4,0x7fffffffe3c8) = 0 (0x0)
20+1 records in
20+1 records out
write(2,"20+1 records in\n20+1 records ou"...,33) = 33 (0x21)


        matthias
--
Matthias Apitz, ✉ [hidden email], http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub
N € I N zur EU!
"Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber.
Für ein soziales und friedliches Europa der Völker." DKP
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Igor Mozolevsky-2
In reply to this post by Ian Lepore-3
On Sat, 20 Apr 2019 at 19:29, Ian Lepore wrote:

>
> On Sat, 2019-04-20 at 18:45 +0100, Igor Mozolevsky wrote:
> > On Sat, 20 Apr 2019 at 18:00, Ian Lepore wrote:
> >
> > <snip>
> > > dd absolutely will fail to copy the last block of the source if it
> > > isn't exactly the blocksize and you didn't specify conv=sync, and
> > > it
> > > will return a zero status when doing so.  It appears you've
> > > convinced
> > > yourself otherwise, but for anyone else reading this thread, be
> > > aware:
> > > conv=sync is required to copy the last part of the source if it's
> > > smaller than the blocksize.
> >
> >
> > Isn't that contrary to the POSIX Spec? Reading the manual [1], it is
> > implied (from STDERR part of the main section) that dd will read and
> > write partial blocks, cf. truncated blocks?..
> >
> > 1. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
> >
> >
>
> Damn, why do people trim away the entire useful context of an email
> thread when replying?  I almost don't want to reply to this because at
> this point all the useful info is gone.

I only trimmed because you were appearing to be speaking generally and
not specifically, so the trimmed part was superfluous.


> In any case, in retrospect, what I said was wrong.  Using conv=sync is
> important when the output is a device that has fixed block sizes where
> you cannot write a partial block.  When the destination is a file where
> short writes work correctly, it will copy the entire source correctly.


Thanks for clarifying, I was starting to think I misread the POSIX
standard when implementing a local version of dd.


--
Igor M.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: bhyve VM stopped to boot after moving virtio disks

Wojciech Puchar-8
In reply to this post by Ian Lepore-3
>> till the end - i see second copy of GPT partition table that was
>> present
>> in VM.
>>
>
> dd absolutely will fail to copy the last block of the source if it
> isn't exactly the blocksize and you didn't specify conv=sync, and it
> will return a zero status when doing so.  It appears you've convinced
strange but i use dd regularly without such behaviour
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"