nanobsd / dd problem?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

nanobsd / dd problem?

Stefan Hegnauer
Hi,

 

I am using freebsd-current (FreeBSD BUILDMASTER 11.0-CURRENT FreeBSD
11.0-CURRENT #0 r259095: Sun Dec  8 10:20:40 CET 2013
root@BUILDMASTER:/usr/obj/usr/src/sys/ASUS  i386) in a VirtualBox as a build
machine for nanobsd images to be used on pc-engines.ch alix boards. The only
difference to GENERIC is the inclusion of 'march=geode' and disabling of
most debugging switches (malloc, Witness etc). Worked like a charm in the
past.

 

Since late summer - sorry, no exact date / svn revision - nanobsd.sh fails
at the last stage when building the disk image, e.g. with

...

00:00:25 ### log: /usr/obj/nanobsd.alixpf//_.di

#

 

Looking a bit closer it seems that dd(1) returns with an I/O error whenever
the input is a file created with mdconfig(8):

# dd if=/dev/zero of=somebackingfile bs=1k count=5k

# mdconfig -f somebackingfile -u md0

# newfs -U /dev/md0

# dd if=/dev/md0 of=/dev/null

dd: /dev/md0: Input/output error

10241+0 records in

10241+0 records out

5243392 bytes transferred in 3.240345 secs (1618159 bytes/sec)

 

The outputfile in nanobsd.sh seems to be error-free.

Anyone else seen similar behaviour? How to proceed/fix it?

 

Thanks

Stefan

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

Olivier Cochard-Labbé-2
On Sun, Dec 8, 2013 at 6:31 PM, Stefan Hegnauer <[hidden email]> wrote:
> Hi,
>
>
> Since late summer - sorry, no exact date / svn revision - nanobsd.sh fails
> at the last stage when building the disk image, e.g. with

Hi, it's a regression in dd:

http://lists.freebsd.org/pipermail/freebsd-current/2013-December/047032.html

Regards,

Olivier
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

Adrian Chadd-2
Which svn rev introduced the regression?

-a

On 8 December 2013 10:19, Olivier Cochard-Labbé <[hidden email]> wrote:

> On Sun, Dec 8, 2013 at 6:31 PM, Stefan Hegnauer <[hidden email]> wrote:
>> Hi,
>>
>>
>> Since late summer - sorry, no exact date / svn revision - nanobsd.sh fails
>> at the last stage when building the disk image, e.g. with
>
> Hi, it's a regression in dd:
>
> http://lists.freebsd.org/pipermail/freebsd-current/2013-December/047032.html
>
> Regards,
>
> Olivier
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

Konstantin Belousov
In reply to this post by Stefan Hegnauer
On Sun, Dec 08, 2013 at 06:31:36PM +0100, Stefan Hegnauer wrote:

> Hi,
>
>  
>
> I am using freebsd-current (FreeBSD BUILDMASTER 11.0-CURRENT FreeBSD
> 11.0-CURRENT #0 r259095: Sun Dec  8 10:20:40 CET 2013
> root@BUILDMASTER:/usr/obj/usr/src/sys/ASUS  i386) in a VirtualBox as a build
> machine for nanobsd images to be used on pc-engines.ch alix boards. The only
> difference to GENERIC is the inclusion of 'march=geode' and disabling of
> most debugging switches (malloc, Witness etc). Worked like a charm in the
> past.
>
>  
>
> Since late summer - sorry, no exact date / svn revision - nanobsd.sh fails
> at the last stage when building the disk image, e.g. with
>
> ...
>
> 00:00:25 ### log: /usr/obj/nanobsd.alixpf//_.di
>
> #
>
>  
>
> Looking a bit closer it seems that dd(1) returns with an I/O error whenever
> the input is a file created with mdconfig(8):
>
> # dd if=/dev/zero of=somebackingfile bs=1k count=5k
>
> # mdconfig -f somebackingfile -u md0
>
> # newfs -U /dev/md0
>
> # dd if=/dev/md0 of=/dev/null
>
> dd: /dev/md0: Input/output error
>
> 10241+0 records in
>
> 10241+0 records out
>
> 5243392 bytes transferred in 3.240345 secs (1618159 bytes/sec)
>
>  
>
> The outputfile in nanobsd.sh seems to be error-free.
It should be one block larger than the right size.
\
>
> Anyone else seen similar behaviour? How to proceed/fix it?
>

The following patch should clear the error.

The issue is that kern_physio() incorrectly detects EOF due to incorrect
calculation of bio bio_resid after the bio_length was clipped by the
'excess' code in g_io_check(). Both bio_length and bio_resid appear
to be 0 in the pre-last dd transfer, which starts exactly and the
mediasize, and kern_physio() thinks that it transferred one more block
than was transferred.

I _suspect_ that it was caused by 'excess' code moving in r256880,
but I am really not in the right condition to analyze it.  If somebody
could try the same dd experiment to confirm or deny my suspicion, it
would be useful.

The patch below should be a right thing to do anyway.

diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c
index c23a74b..b7c4d60 100644
--- a/sys/kern/vfs_bio.c
+++ b/sys/kern/vfs_bio.c
@@ -3679,7 +3679,6 @@ bufdonebio(struct bio *bip)
 
  bp = bip->bio_caller2;
  bp->b_resid = bp->b_bcount - bip->bio_completed;
- bp->b_resid = bip->bio_resid; /* XXX: remove */
  bp->b_ioflags = bip->bio_flags;
  bp->b_error = bip->bio_error;
  if (bp->b_error)

attachment0 (851 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: nanobsd / dd problem?

Stefan Hegnauer
On Monday, December 09, 2013 at 5:43 AM Konstantin Belousov wrote:


> On Sun, Dec 08, 2013 at 06:31:36PM +0100, Stefan Hegnauer wrote:
> > Hi,
> >
> >
> >
> > I am using freebsd-current (FreeBSD BUILDMASTER 11.0-CURRENT FreeBSD
> > 11.0-CURRENT #0 r259095: Sun Dec  8 10:20:40 CET 2013
> > root@BUILDMASTER:/usr/obj/usr/src/sys/ASUS  i386) in a VirtualBox as
> a build
> > machine for nanobsd images to be used on pc-engines.ch alix boards.
> The only
> > difference to GENERIC is the inclusion of 'march=geode' and disabling
> of
> > most debugging switches (malloc, Witness etc). Worked like a charm in
> the
> > past.
> >
> >
> >
> > Since late summer - sorry, no exact date / svn revision - nanobsd.sh
> fails
> > at the last stage when building the disk image, e.g. with
> >
> > ...
> >
> > 00:00:25 ### log: /usr/obj/nanobsd.alixpf//_.di
> >
> > #
> >
> >
> >
> > Looking a bit closer it seems that dd(1) returns with an I/O error
> whenever
> > the input is a file created with mdconfig(8):
> >
> > # dd if=/dev/zero of=somebackingfile bs=1k count=5k
> >
> > # mdconfig -f somebackingfile -u md0
> >
> > # newfs -U /dev/md0
> >
> > # dd if=/dev/md0 of=/dev/null
> >
> > dd: /dev/md0: Input/output error
> >
> > 10241+0 records in
> >
> > 10241+0 records out
> >
> > 5243392 bytes transferred in 3.240345 secs (1618159 bytes/sec)
> >
> >
> >
> > The outputfile in nanobsd.sh seems to be error-free.
> It should be one block larger than the right size.
> \
> >
> > Anyone else seen similar behaviour? How to proceed/fix it?
> >
>
> The following patch should clear the error.
>
> The issue is that kern_physio() incorrectly detects EOF due to
> incorrect
> calculation of bio bio_resid after the bio_length was clipped by the
> 'excess' code in g_io_check(). Both bio_length and bio_resid appear
> to be 0 in the pre-last dd transfer, which starts exactly and the
> mediasize, and kern_physio() thinks that it transferred one more block
> than was transferred.
>
> I _suspect_ that it was caused by 'excess' code moving in r256880,
> but I am really not in the right condition to analyze it.  If somebody
> could try the same dd experiment to confirm or deny my suspicion, it
> would be useful.
>
> The patch below should be a right thing to do anyway.
>
> diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c
> index c23a74b..b7c4d60 100644
> --- a/sys/kern/vfs_bio.c
> +++ b/sys/kern/vfs_bio.c
> @@ -3679,7 +3679,6 @@ bufdonebio(struct bio *bip)
>
>   bp = bip->bio_caller2;
>   bp->b_resid = bp->b_bcount - bip->bio_completed;
> - bp->b_resid = bip->bio_resid; /* XXX: remove */
>   bp->b_ioflags = bip->bio_flags;
>   bp->b_error = bip->bio_error;
>   if (bp->b_error)

Works for me - please commit!
Thanks a lot!

-Stefan

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

freebsdonline
Stefan Hegnauer wrote:

> On Monday, December 09, 2013 at 5:43 AM Konstantin Belousov wrote:
>
>
>> On Sun, Dec 08, 2013 at 06:31:36PM +0100, Stefan Hegnauer wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am using freebsd-current (FreeBSD BUILDMASTER 11.0-CURRENT FreeBSD
>>> 11.0-CURRENT #0 r259095: Sun Dec  8 10:20:40 CET 2013
>>> root@BUILDMASTER:/usr/obj/usr/src/sys/ASUS  i386) in a VirtualBox as
>> a build
>>> machine for nanobsd images to be used on pc-engines.ch alix boards.
>> The only
>>> difference to GENERIC is the inclusion of 'march=geode' and disabling
>> of
>>> most debugging switches (malloc, Witness etc). Worked like a charm in
>> the
>>> past.
>>>
>>>
>>>
>>> Since late summer - sorry, no exact date / svn revision - nanobsd.sh
>> fails
>>> at the last stage when building the disk image, e.g. with
>>>
>>> ...
>>>
>>> 00:00:25 ### log: /usr/obj/nanobsd.alixpf//_.di
>>>
>>> #
>>>
>>>
>>>
>>> Looking a bit closer it seems that dd(1) returns with an I/O error
>> whenever
>>> the input is a file created with mdconfig(8):
>>>
>>> # dd if=/dev/zero of=somebackingfile bs=1k count=5k
>>>
>>> # mdconfig -f somebackingfile -u md0
>>>
>>> # newfs -U /dev/md0
>>>
>>> # dd if=/dev/md0 of=/dev/null
>>>
>>> dd: /dev/md0: Input/output error
>>>
>>> 10241+0 records in
>>>
>>> 10241+0 records out
>>>
>>> 5243392 bytes transferred in 3.240345 secs (1618159 bytes/sec)
>>>
>>>
>>>
>>> The outputfile in nanobsd.sh seems to be error-free.
>> It should be one block larger than the right size.
>> \
>>> Anyone else seen similar behaviour? How to proceed/fix it?
>>>
>> The following patch should clear the error.
>>
>> The issue is that kern_physio() incorrectly detects EOF due to
>> incorrect
>> calculation of bio bio_resid after the bio_length was clipped by the
>> 'excess' code in g_io_check(). Both bio_length and bio_resid appear
>> to be 0 in the pre-last dd transfer, which starts exactly and the
>> mediasize, and kern_physio() thinks that it transferred one more block
>> than was transferred.
>>
>> I _suspect_ that it was caused by 'excess' code moving in r256880,
>> but I am really not in the right condition to analyze it.  If somebody
>> could try the same dd experiment to confirm or deny my suspicion, it
>> would be useful.
>>
>> The patch below should be a right thing to do anyway.
>>
>> diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c
>> index c23a74b..b7c4d60 100644
>> --- a/sys/kern/vfs_bio.c
>> +++ b/sys/kern/vfs_bio.c
>> @@ -3679,7 +3679,6 @@ bufdonebio(struct bio *bip)
>>
>>   bp = bip->bio_caller2;
>>   bp->b_resid = bp->b_bcount - bip->bio_completed;
>> - bp->b_resid = bip->bio_resid; /* XXX: remove */
>>   bp->b_ioflags = bip->bio_flags;
>>   bp->b_error = bip->bio_error;
>>   if (bp->b_error)
> Works for me - please commit!
> Thanks a lot!
>
> -Stefan
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[hidden email]"
>
I have a similar problem with nanobsd.sh on FreeBSD 10.x. I am able to
build the image but sometimes (could not figured out the reason why) the
image is build incorrectly.

For example on an incorrectly built image when I boot i got:

Trying to mount root from ufs:/dev/ada0s1a [ro]...
mount: /dev/ada0s3: Invalid argument
mount -o ro /dev/ada0s3 /conf/default/etc failed: dropping into /bin/sh

It seems the partition that hold config files cannot be accessed.

Hope your fix will fix this problem too (asumming you will also commit
on 10.x).

Theres also an issue with package management, the nanobsd script still
uses pkg_add instead of pkg.

ovi

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

Peter Holm-3
In reply to this post by Konstantin Belousov
On Mon, Dec 09, 2013 at 06:42:39AM +0200, Konstantin Belousov wrote:

> On Sun, Dec 08, 2013 at 06:31:36PM +0100, Stefan Hegnauer wrote:
> > Hi,
> >
> >  
> >
> > I am using freebsd-current (FreeBSD BUILDMASTER 11.0-CURRENT FreeBSD
> > 11.0-CURRENT #0 r259095: Sun Dec  8 10:20:40 CET 2013
> > root@BUILDMASTER:/usr/obj/usr/src/sys/ASUS  i386) in a VirtualBox as a build
> > machine for nanobsd images to be used on pc-engines.ch alix boards. The only
> > difference to GENERIC is the inclusion of 'march=geode' and disabling of
> > most debugging switches (malloc, Witness etc). Worked like a charm in the
> > past.
> >
> >  
> >
> > Since late summer - sorry, no exact date / svn revision - nanobsd.sh fails
> > at the last stage when building the disk image, e.g. with
> >
> > ...
> >
> > 00:00:25 ### log: /usr/obj/nanobsd.alixpf//_.di
> >
> > #
> >
> >  
> >
> > Looking a bit closer it seems that dd(1) returns with an I/O error whenever
> > the input is a file created with mdconfig(8):
> >
> > # dd if=/dev/zero of=somebackingfile bs=1k count=5k
> >
> > # mdconfig -f somebackingfile -u md0
> >
> > # newfs -U /dev/md0
> >
> > # dd if=/dev/md0 of=/dev/null
> >
> > dd: /dev/md0: Input/output error
> >
> > 10241+0 records in
> >
> > 10241+0 records out
> >
> > 5243392 bytes transferred in 3.240345 secs (1618159 bytes/sec)
> >
> >  
> >
> > The outputfile in nanobsd.sh seems to be error-free.
> It should be one block larger than the right size.
> \
> >
> > Anyone else seen similar behaviour? How to proceed/fix it?
> >
>
> The following patch should clear the error.
>
> The issue is that kern_physio() incorrectly detects EOF due to incorrect
> calculation of bio bio_resid after the bio_length was clipped by the
> 'excess' code in g_io_check(). Both bio_length and bio_resid appear
> to be 0 in the pre-last dd transfer, which starts exactly and the
> mediasize, and kern_physio() thinks that it transferred one more block
> than was transferred.
>
> I _suspect_ that it was caused by 'excess' code moving in r256880,
> but I am really not in the right condition to analyze it.  If somebody
> could try the same dd experiment to confirm or deny my suspicion, it
> would be useful.
>
> The patch below should be a right thing to do anyway.
>
> diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c
> index c23a74b..b7c4d60 100644
> --- a/sys/kern/vfs_bio.c
> +++ b/sys/kern/vfs_bio.c
> @@ -3679,7 +3679,6 @@ bufdonebio(struct bio *bip)
>  
>   bp = bip->bio_caller2;
>   bp->b_resid = bp->b_bcount - bip->bio_completed;
> - bp->b_resid = bip->bio_resid; /* XXX: remove */
>   bp->b_ioflags = bip->bio_flags;
>   bp->b_error = bip->bio_error;
>   if (bp->b_error)

I have tested this patch with a buildworld + selected other tests. No
problems seen (and problem fixed, of cause).

- Peter
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

Olivier Cochard-Labbé-2
In reply to this post by Konstantin Belousov
On Mon, Dec 9, 2013 at 5:42 AM, Konstantin Belousov <[hidden email]>wrote:

>
>
>
> The patch below should be a right thing to do anyway.
>
> diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c
> index c23a74b..b7c4d60 100644
> --- a/sys/kern/vfs_bio.c
> +++ b/sys/kern/vfs_bio.c
> @@ -3679,7 +3679,6 @@ bufdonebio(struct bio *bip)
>
>         bp = bip->bio_caller2;
>         bp->b_resid = bp->b_bcount - bip->bio_completed;
> -       bp->b_resid = bip->bio_resid;   /* XXX: remove */
>         bp->b_ioflags = bip->bio_flags;
>         bp->b_error = bip->bio_error;
>         if (bp->b_error)
>

Problem fixed with your patch for me too: Thanks.

Olivier
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

mike-6
In reply to this post by freebsdonline
Hi,

On Mon, 9 Dec 2013, freebsdonline wrote:
...
> Theres also an issue with package management, the nanobsd script still uses
> pkg_add instead of pkg.
>

for -current I've added cust_pkgng() to nanobsd.sh
It needs to get called manually instead of cust_pkg() though.
( customize_cmd cust_pkgng )

Greetings
---
Michael Reifenberger
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: nanobsd / dd problem?

freebsdonline
mike wrote:

> Hi,
>
> On Mon, 9 Dec 2013, freebsdonline wrote:
> ...
>> Theres also an issue with package management, the nanobsd script
>> still uses pkg_add instead of pkg.
>>
>
> for -current I've added cust_pkgng() to nanobsd.sh
> It needs to get called manually instead of cust_pkg() though.
> ( customize_cmd cust_pkgng )
>
> Greetings
> ---
> Michael Reifenberger
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to
> "[hidden email]"
>
Still the nanobsd.sh script is building a broken image on FreeBSD 10.0 RC4:

After booting the image in a VirtualBox machine:

Trying to mount root from ufs:/dev/ada0s1a [ro]...
mount: /dev/ada0s3: Invalid argument
mount -o ro /dev/ada0s3 /conf/default/etc failed: dropping into /bin/sh

It seems the third partition (the one that holds the data for config
files) has errors on it:

# disklabel /dev/ada0s3
disklabel /dev/ada0s3
disklabel: /dev/ada0s3: no valid label found

In my nanobsd.conf I have (20MB for data partition):

NANO_MEDIASIZE=2097152
NANO_DATASIZE=40960
NANO_IMAGES=2

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"