Trouble using and understanding funopen(3)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Trouble using and understanding funopen(3)

freebsd-hackers mailing list
Hi all,

I am having trouble understanding how funopen(3)'s read function works. Specifically, how do I have the readfn return with less than the requested amount of bytes.

My understanding: I believe that funopen(3) allows you to assign the read, write and close methods to a FILE stream. When a program runs fread(3) on a FILE stream opened by funopen(3), the program will run the readfn handler in a loop until it returns either returns the requested number of bytes, 0 or -1 (error).

Question: How do I structure the code so that readfn returns with less than the numbe of requested bytes? For example, what if the calling fread() function requests 100 bytes, but the readfn can only return 10 bytes? What mechanism do I need to implement so that the fread(3) returns "10" bytes rather than the readfn handler running 10 times? This results in the fread()'s return value as 100, even though only 10 bytes were *actually* read.

I have looked at a few examples from the src tree. Clearly they have to use buffering and append the bytes they read to the memory object they were initially passed. Somehow they return with the number of bytes they actually read, not necessarily the requested amount. But it is not clear to me how they make this distinction and avoid having their respective readfn function re-rerun. Also, in the examples I did look up there does not appear to be any use of setvbuf().

Below is a very simple test case to illustrate the issue.

------
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

static int
ssh_readfn(void *v, char *buf, int len)
{
        printf("Running readfn handler\n");
        memcpy(buf, "AAAAAAAAAA", 10);
        return 10;
}

static int
ssh_writefn(void *v, const char *buf, int len)
{
        return 0;
}

int
main()
{
        int x;
        char buf[1000];
        FILE *f;

        f = funopen(NULL, ssh_readfn, ssh_writefn, NULL, NULL);
        if (f == NULL) {
                printf("funopen failed, exiting.\n");
                exit(0);
        }

        x = fread(buf, 1, 100, f);
        printf("Bytes read: %d\n", x);
}
------

This displays 10 "Running readfn handler" lines fllowed by "Bytes read: 100" even though I am explicitly returning 10 in ssh_readfn. Please advise what the mechanism is only return with less than the requested number of bytes.

Thanks!
---
Farhan Khan
PGP Fingerprint: 1312 89CE 663E 1EB2 179C 1C83 C41D 2281 F8DA C0DE
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Trouble using and understanding funopen(3)

Conrad Meyer-2
Hi Farhan,

First, I'd suggest using the more portable fopencookie(3) interface,
which is similar to funopen(3).

Second, read functions return 0 to indicate end of file.

Finally, the file cookie routines are stateful.  If you want to create
a pseudo-FILE that only has 10 bytes in it, you have to track the
current file offset by creating a cookie.  Here is a minimal example
of using a cookie.

struct my_file {
    off_t offset;
};

my_read(void *v, buf, len)
{
    struct my_file *f = v;
    size_t rlen;

    /* Indicate EOF for reads past EOF. */
    if (f->offset >= 10)
        return (0);

    rlen = MIN(len, 10 - f->offset);
    memcpy(buf, "AAAAAAAAAA", rlen);
    f->offset += rlen;

    return ((int)rlen);
}

main()
{
    struct my_file *cookie;
    FILE *f;
    char buf[100];
    size_t x;

    cookie = calloc(1, sizeof(*cookie));
    f = fopencookie(cookie, "rb", { .read = my_read, });

    x = fread(buf, 1, sizeof(buf), f);
    ...
}

Conrad


On Thu, Aug 22, 2019 at 9:24 AM Farhan Khan via freebsd-hackers
<[hidden email]> wrote:

>
> Hi all,
>
> I am having trouble understanding how funopen(3)'s read function works. Specifically, how do I have the readfn return with less than the requested amount of bytes.
>
> My understanding: I believe that funopen(3) allows you to assign the read, write and close methods to a FILE stream. When a program runs fread(3) on a FILE stream opened by funopen(3), the program will run the readfn handler in a loop until it returns either returns the requested number of bytes, 0 or -1 (error).
>
> Question: How do I structure the code so that readfn returns with less than the numbe of requested bytes? For example, what if the calling fread() function requests 100 bytes, but the readfn can only return 10 bytes? What mechanism do I need to implement so that the fread(3) returns "10" bytes rather than the readfn handler running 10 times? This results in the fread()'s return value as 100, even though only 10 bytes were *actually* read.
>
> I have looked at a few examples from the src tree. Clearly they have to use buffering and append the bytes they read to the memory object they were initially passed. Somehow they return with the number of bytes they actually read, not necessarily the requested amount. But it is not clear to me how they make this distinction and avoid having their respective readfn function re-rerun. Also, in the examples I did look up there does not appear to be any use of setvbuf().
>
> Below is a very simple test case to illustrate the issue.
>
> ------
> #include <stdio.h>
> #include <string.h>
> #include <stdlib.h>
>
> static int
> ssh_readfn(void *v, char *buf, int len)
> {
>         printf("Running readfn handler\n");
>         memcpy(buf, "AAAAAAAAAA", 10);
>         return 10;
> }
>
> static int
> ssh_writefn(void *v, const char *buf, int len)
> {
>         return 0;
> }
>
> int
> main()
> {
>         int x;
>         char buf[1000];
>         FILE *f;
>
>         f = funopen(NULL, ssh_readfn, ssh_writefn, NULL, NULL);
>         if (f == NULL) {
>                 printf("funopen failed, exiting.\n");
>                 exit(0);
>         }
>
>         x = fread(buf, 1, 100, f);
>         printf("Bytes read: %d\n", x);
> }
> ------
>
> This displays 10 "Running readfn handler" lines fllowed by "Bytes read: 100" even though I am explicitly returning 10 in ssh_readfn. Please advise what the mechanism is only return with less than the requested number of bytes.
>
> Thanks!
> ---
> Farhan Khan
> PGP Fingerprint: 1312 89CE 663E 1EB2 179C 1C83 C41D 2281 F8DA C0DE
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Trouble using and understanding funopen(3)

freebsd-hackers mailing list
August 22, 2019 4:28 PM, "Conrad Meyer" <[hidden email]> wrote:

> Hi Farhan,
>
> First, I'd suggest using the more portable fopencookie(3) interface,
> which is similar to funopen(3).
>
> Second, read functions return 0 to indicate end of file.
>
> Finally, the file cookie routines are stateful. If you want to create
> a pseudo-FILE that only has 10 bytes in it, you have to track the
> current file offset by creating a cookie. Here is a minimal example
> of using a cookie.
>
> struct my_file {
> off_t offset;
> };
>
> my_read(void *v, buf, len)
> {
> struct my_file *f = v;
> size_t rlen;
>
> /* Indicate EOF for reads past EOF. */
> if (f->offset >= 10)
> return (0);
>
> rlen = MIN(len, 10 - f->offset);
> memcpy(buf, "AAAAAAAAAA", rlen);
> f->offset += rlen;
>
> return ((int)rlen);
> }
>
> main()
> {
> struct my_file *cookie;
> FILE *f;
> char buf[100];
> size_t x;
>
> cookie = calloc(1, sizeof(*cookie));
> f = fopencookie(cookie, "rb", { .read = my_read, });
>
> x = fread(buf, 1, sizeof(buf), f);
> ...
> }
>
> Conrad
>
> On Thu, Aug 22, 2019 at 9:24 AM Farhan Khan via freebsd-hackers
> <[hidden email]> wrote:
>

Conrad,

Took me a while to massage that into the format I wanted, but I got it working. Thanks!

To be honest, I had trouble understanding the manual. I was mistakenly under the impression that the readfn function was literally just a pass-through to whatever read mechanism you implement. It also was not obvious how the return value is interpreted. Do you think it might be useful to get an update to the documentation? Or was this me failing to understand the way funopen worked?

Also, Linux's fopencookie(3) has a code sample. That might be good to have as well.

Thanks

---
Farhan Khan
PGP Fingerprint: 1312 89CE 663E 1EB2 179C  1C83 C41D 2281 F8DA C0DE
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"