Сообщение Re: Обмен данными между "Virtualbox guest OS COM port" and " от 01.11.2022 11:14
Изменено 01.11.2022 11:43 grizlyk1
Re: Обмен данными между "Virtualbox guest OS COM port" and "
************
Часть вторая
************
Let's continue to explore <stdio.h> stream access.
The original program did not work better because only a few errors returned by system calls was checked by assert() in the program.
If we check more returns we got more error messages, we got error messages on every wrong fputc/fgetc, that means stdio knows about the errors, but only we do not.
The improved version of initial example,
example2
We could collect the error messages by more asserts, but the messages could not help us to understand "what the error is", because when we tried to access //./pipe/* from client side we did assume:
— we must not fseek() on pipes (will should always get ESPIPE error on every fseek());
— we can not violate OS quotas for number of pipe client side opened handlers (virtualbox pipe server side declares the quotas).
But in real <stdio.h> life, we not only "can", we "must", we should do assume:
— we must always DO fseek() on pipes when interleave read/write requests (otherwise will always get fputc/fgetc error);
— we can violate OS quotas for "number of pipe client side opened handlers" by dup() to create "rb"+"ab" separated access handlers pair;
— we must know dup() can not "elevate up" RW access of handler created by open() (dup() can only more restrict the existing RW access created by open()).
It is ridiculous, the real <stdio.h> conditions are strictly opposit to our expectations, and we must find why.
1.1
The OS quotas means we can not do like this
Due to the OS quotas we must use "r+" mode to r/w by one bi-directional handler
As we know, windows has no posix "named FIFOs", means windows kernel does not provide "standard pipe server side" to any number of pipe clients, who has permissions defined by the file system chmod/chown attrs to connect pipe.
In our case virtualbox provides own access rules of own pipe server side for "//./pipe/dos1" and the 'standard rules' for windows is 'single client only'.
1.2
One user told us abstract answer: "dup this to anything", so we can write a code to create two separated single_direction streams fcr/fcw.
The splitted fcr/fcw streams have no "r+" read/write interleave problems of original bi-directional fc stream.
Mode "a"/"w" is often not the same we need from "w" direction, so we use "r+" as "w" direction and to avoid r/w interleave problems just does not read from "r+".
1.3
The next improved version of initial example
example3
Now the example looks better, there are no more:
— fseek on pipe;
— fseek on every write to local file;
— fseek two times/per loop pass on every r/w interleave for pipe.
2.
There are some questions about <stdlib.h>.
About dup() behaviour.
Due to the guess about the dup() behaviour we can explain why:
— dup() can violate OS quotas for number of opened handlers;
— dup() can not "elevate up" RW access of system_handler created by open() (dup() can only more restrict the existing RW access created by open()).
It looks like process handler is abstract structure { system_handler data, local_handler data}.
open() creates both members in the abstract structure, but dup can change only local_handler field, copying the same system_handler field to new handler.
That means that from OS point of view, all duped handlers does the same access via the same system_handler, as if we several times called fread() for the same system_handler directly.
3.
Why we must not do fseek() on pipes?
Unrelated to stdio rules, we can not fseek() on devices which are incapable of seeking because there are no file pointer here, the devices are represented as pair {input_port, output_port}:
— there is no position of data inside each IO direction (not exist "input position" or "output position");
— there is no position of data between different IO directions ("input_port position" is not less, more or equal to "output position");
there are many devices that follow the restrictions.
And the "posix prog man" claims
— "for fseek() the value of the file offset on devices which are incapable of seeking is undefined".
that means stdio is ignoring {offset,whence} parameters of fseek() call on FILE which are incapable of seeking and fseek(fc, rand(), rand()) also will work in our example2 instead of assert( !fseek(fc,0,0) ).
The "posix prog man" claims
— "the behavior of fseek() on devices which are incapable of seeking is implementation-defined"
that means we must not call fseek() on devices which are incapable of seeking, fseek() must be used for directly purpose only, in order to set file pointer and not for any other jobs.
3.1
The question is ability to subst regular file instead of trivial pipe in the same program.
In order to work in program with regular file as if the file is a trivial pipe, we can restrict in the program fseek() usage to cases:
sequential read file starting from begin of file fseek(fi,0,SEEK_SET);
sequential write file by append after end of file fseek(fo,0,SEEK_END).
That means pipe should "defines up to complete" the prohibited fseek() by the two allowed fseek() calls with parameters '(0,SEEK_SET)/(0,SEEK_END)'.
The fseek() parameters have no sense for pipe but provide compatibility of pipe with regular file (SEEK_SET refer to read pipe, SEEK_END refer to write pipe).
sure, the purpose of stdio is to avoid us from dig the stream details and guess stream attr combinations, i have never looked so closely before.
3.2
We can imagine stdio based program, that does random access to regular file by fseek(offset), fread(size)/fwrite(size). Later we decided to feed the program by pipe instead of regular file, using external software created "gathered" pipe with "chuncks" of the same (offset,size).
In the case we need stdio behaviour is ignoring {offset, whence} parameters of fseek() call on pipe.
But in real the similar "gathered" pipe is just improper use of pipe, the programming technic produces unreliable code and leads to runtime errors, just believe that runtime errors is one of the most unpleasure thing you want to get.
Improper in comparison with "mapping regular file to pipe" that is implemented by only two special cases of fseek(fi,0,SEEK_SET)/fseek(fi,0,SEEK_END) calls allowed for pipe.
4.
why we need fseek() for r/w interleave in buffered "r+" mode?
FILE provides single buffer to IO and read/write data from downlayer handler by "chunks" into the buffer (reducing IO system calls).
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
but when in "r+" mode we change IO direction (read->write or write->read) we need flush dirty write buffer and drop (invalidate by LRU rules) obsolete read buffer.
But flush only is not enough, most time pointer in FILE and pointer in downlayer handler are not the same, we need fseek to change IO direction.
consider
"on devices which are incapable of seeking" in buffered "r+" mode just split pipe by fdup_r() or disable buffering by setvbuf(FILE, 0, _IONBF, 0).
5.
As result.
Once time earlier we have claimed "posix provides reliable way to access streams", but now we see this was not really true, there are some issues here.
And i have never seen the same info about dup() and 'r+' mode in "prog references".
Часть вторая
************
Let's continue to explore <stdio.h> stream access.
The original program did not work better because only a few errors returned by system calls was checked by assert() in the program.
If we check more returns we got more error messages, we got error messages on every wrong fputc/fgetc, that means stdio knows about the errors, but only we do not.
int
//fgetc emits pipe error if there was no fseek
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
...
//fputc emits pipe error if there was no fseek
assert( fputc(tx_ch,fc) != EOF );
...The improved version of initial example,
example2
enum{ PR_SYN= 0x16U };
fc= fopen("//./pipe/dos1","rb+"); assert(fc);
//unbuffered is always required for selected AUX port protocol "per char" exchange
assert( !setvbuf(fc, 0, _IONBF, 0) );
fi= fopen("./isw","rb"); assert(fi);
fo= fopen("./osw","ab"); assert(fo);
//
for(;;){
//rx
assert( !fseek(fc,0,SEEK_SET) );
int
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
assert( !fseek(fc,0,SEEK_END) );
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fc) != EOF );
}We could collect the error messages by more asserts, but the messages could not help us to understand "what the error is", because when we tried to access //./pipe/* from client side we did assume:
— we must not fseek() on pipes (will should always get ESPIPE error on every fseek());
— we can not violate OS quotas for number of pipe client side opened handlers (virtualbox pipe server side declares the quotas).
But in real <stdio.h> life, we not only "can", we "must", we should do assume:
— we must always DO fseek() on pipes when interleave read/write requests (otherwise will always get fputc/fgetc error);
— we can violate OS quotas for "number of pipe client side opened handlers" by dup() to create "rb"+"ab" separated access handlers pair;
— we must know dup() can not "elevate up" RW access of handler created by open() (dup() can only more restrict the existing RW access created by open()).
It is ridiculous, the real <stdio.h> conditions are strictly opposit to our expectations, and we must find why.
1.1
The OS quotas means we can not do like this
fcr= fopen("//./pipe/dos1", "rb"); assert(fcr);
fcw= fopen("//./pipe/dos1", "ab"); assert(fcw); //fcw always errorDue to the OS quotas we must use "r+" mode to r/w by one bi-directional handler
fc= fopen("//./pipe/dos1", "rb+"); assert(fc);As we know, windows has no posix "named FIFOs", means windows kernel does not provide "standard pipe server side" to any number of pipe clients, who has permissions defined by the file system chmod/chown attrs to connect pipe.
In our case virtualbox provides own access rules of own pipe server side for "//./pipe/dos1" and the 'standard rules' for windows is 'single client only'.
1.2
One user told us abstract answer: "dup this to anything", so we can write a code to create two separated single_direction streams fcr/fcw.
The splitted fcr/fcw streams have no "r+" read/write interleave problems of original bi-directional fc stream.
Mode "a"/"w" is often not the same we need from "w" direction, so we use "r+" as "w" direction and to avoid r/w interleave problems just does not read from "r+".
{
//here regular job for 'rb+' access
//splitted FILE can be buffered
//this is program responsibility to undestand the r/w buffers are shared or separated
//and provide coherence of multiple r/w shared buffers
//assert( !setvbuf(fc, 0, _IONBF, 0) );
//must be created in 'r+' access (O_RDWR)
//the original "rb+" will be used for "w" direction (O_WRONLY)
//dup() also can not "elevate up" RW attr, can not create "w" direction (O_WRONLY) from "r" direction (O_RDONLY)
fcw= fopen("//./pipe/dos1", "rb+"); assert(fcw);
//here function to do the FILE split regular job for 'rb+' access
{
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
fcr= fdopen(dfcr, "rb"); assert(fcr);
//drop dfc, dfcr values
}
//we can declare function
//to do the FILE split regular job for 'rb+' access
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(FILE *const fcw);
}1.3
The next improved version of initial example
example3
//
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(
FILE *const fcw
){
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
//fcr
return fdopen(dfcr, "rb");
//drop dfc, dfcr values
}
//
{
enum{ PR_SYN= 0x16U };
//pipe
//open pipe in "r+"("w") mode fcw
fcw= fopen("//./pipe/dos1","rb+"); assert(fcw);
//return "r" mode dupped fcr from original "r+"("w") mode fcw
fcr= fdup_r(fcw); assert(fcr);
//unbuffered pipe is always required for our AUX port protocol "per char" exchange
assert( !setvbuf(fcr, 0, _IONBF, 0) );
assert( !setvbuf(fcw, 0, _IONBF, 0) );
//allow subst regular file instead of pipe
assert( !fseek(fcr,0,SEEK_SET) );
assert( !fseek(fcw,0,SEEK_END) );
//local files
fi= fopen("./isw","rb"); assert(fi);
//append + reopen to r+
fo= fopen("./osw","ab+"); assert(fo); fclose(fo);
fo= fopen("./osw","rb+"); assert(fo); assert( !fseek(fo,0,SEEK_END) );
//exchange loop
for(;;){
//rx
int
rx_ch= fgetc(fcr);
if(rx_ch == EOF){ assert( feof(fcr) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fcw) != EOF );
}}Now the example looks better, there are no more:
— fseek on pipe;
— fseek on every write to local file;
— fseek two times/per loop pass on every r/w interleave for pipe.
2.
There are some questions about <stdlib.h>.
About dup() behaviour.
Due to the guess about the dup() behaviour we can explain why:
— dup() can violate OS quotas for number of opened handlers;
— dup() can not "elevate up" RW access of system_handler created by open() (dup() can only more restrict the existing RW access created by open()).
It looks like process handler is abstract structure { system_handler data, local_handler data}.
open() creates both members in the abstract structure, but dup can change only local_handler field, copying the same system_handler field to new handler.
That means that from OS point of view, all duped handlers does the same access via the same system_handler, as if we several times called fread() for the same system_handler directly.
3.
Why we must not do fseek() on pipes?
Unrelated to stdio rules, we can not fseek() on devices which are incapable of seeking because there are no file pointer here, the devices are represented as pair {input_port, output_port}:
— there is no position of data inside each IO direction (not exist "input position" or "output position");
— there is no position of data between different IO directions ("input_port position" is not less, more or equal to "output position");
there are many devices that follow the restrictions.
And the "posix prog man" claims
— "for fseek() the value of the file offset on devices which are incapable of seeking is undefined".
that means stdio is ignoring {offset,whence} parameters of fseek() call on FILE which are incapable of seeking and fseek(fc, rand(), rand()) also will work in our example2 instead of assert( !fseek(fc,0,0) ).
The "posix prog man" claims
— "the behavior of fseek() on devices which are incapable of seeking is implementation-defined"
that means we must not call fseek() on devices which are incapable of seeking, fseek() must be used for directly purpose only, in order to set file pointer and not for any other jobs.
3.1
The question is ability to subst regular file instead of trivial pipe in the same program.
In order to work in program with regular file as if the file is a trivial pipe, we can restrict in the program fseek() usage to cases:
sequential read file starting from begin of file fseek(fi,0,SEEK_SET);
sequential write file by append after end of file fseek(fo,0,SEEK_END).
That means pipe should "defines up to complete" the prohibited fseek() by the two allowed fseek() calls with parameters '(0,SEEK_SET)/(0,SEEK_END)'.
The fseek() parameters have no sense for pipe but provide compatibility of pipe with regular file (SEEK_SET refer to read pipe, SEEK_END refer to write pipe).
sure, the purpose of stdio is to avoid us from dig the stream details and guess stream attr combinations, i have never looked so closely before.
3.2
We can imagine stdio based program, that does random access to regular file by fseek(offset), fread(size)/fwrite(size). Later we decided to feed the program by pipe instead of regular file, using external software created "gathered" pipe with "chuncks" of the same (offset,size).
In the case we need stdio behaviour is ignoring {offset, whence} parameters of fseek() call on pipe.
But in real the similar "gathered" pipe is just improper use of pipe, the programming technic produces unreliable code and leads to runtime errors, just believe that runtime errors is one of the most unpleasure thing you want to get.
Improper in comparison with "mapping regular file to pipe" that is implemented by only two special cases of fseek(fi,0,SEEK_SET)/fseek(fi,0,SEEK_END) calls allowed for pipe.
4.
why we need fseek() for r/w interleave in buffered "r+" mode?
FILE provides single buffer to IO and read/write data from downlayer handler by "chunks" into the buffer (reducing IO system calls).
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
but when in "r+" mode we change IO direction (read->write or write->read) we need flush dirty write buffer and drop (invalidate by LRU rules) obsolete read buffer.
But flush only is not enough, most time pointer in FILE and pointer in downlayer handler are not the same, we need fseek to change IO direction.
consider
//read
fseek(0,0)
seek (0,0)
//here FILE and downlayer handler both points to 0
fgetc
//read into buf
read(0,4K)
//here FILE points to buf[1]
//and downlayer handler points to file[4K]
//char was returned from FILE buf[0]
//write
fseek(100,0)
//drop input buf
//here previous input [1..4K) will be dropped
//!HERE "on devices which are incapable of seeking" we can not fseek
//all dropped data will be lost, we can not "push back" the read data into pipe
//input stream data will be damaged just because we try write after read
//"on devices which are incapable of seeking" just split pipe by `fdup_r()`
//or disable buffering by `setvbuf(FILE, 0, _IONBF, 0)`
//seek new write pos 100
seek (100,0)
//here FILE and downlayer handler both points to 100
fputc
//write into buf
//here FILE points to 101
//char was placed into FILE buf[0]
//here downlayer handler points to 100
//write(100,1) was not called"on devices which are incapable of seeking" in buffered "r+" mode just split pipe by fdup_r() or disable buffering by setvbuf(FILE, 0, _IONBF, 0).
5.
As result.
Once time earlier we have claimed "posix provides reliable way to access streams", but now we see this was not really true, there are some issues here.
And i have never seen the same info about dup() and 'r+' mode in "prog references".
Re: Обмен данными между "Virtualbox guest OS COM port" and "
************
Часть вторая
************
Let's continue to explore <stdio.h> stream access.
The original program did not work better because only a few errors returned by system calls was checked by assert() in the program.
If we check more returns we got more error messages, we got error messages on every wrong fputc/fgetc, that means stdio knows about the errors, but only we do not.
The improved version of initial example,
example2
We could collect the error messages by more asserts, but the messages could not help us to understand "what the error is", because when we tried to access //./pipe/* from client side we did assume:
— we must not fseek() on pipes (will should always get ESPIPE error on every fseek());
— we can not violate OS quotas for number of pipe client side opened handlers (virtualbox pipe server side declares the quotas).
But in real <stdio.h> life, we not only "can", we "must", we should do assume:
— we must always DO fseek() on pipes when interleave read/write requests (otherwise will always get fputc/fgetc error);
— we can violate OS quotas for "number of pipe client side opened handlers" by dup() to create "rb"+"ab" separated access handlers pair;
— we must know dup() can not "elevate up" RW access of handler created by open() (dup() can only more restrict the existing RW access created by open()).
It is ridiculous, the real <stdio.h> conditions are strictly opposit to our expectations, and we must find why.
1.1
The OS quotas means we can not do like this
Due to the OS quotas we must use "r+" mode to r/w by one bi-directional handler
As we know, windows has no posix "named FIFOs", means windows kernel does not provide "standard pipe server side" to any number of pipe clients, who has permissions defined by the file system chmod/chown attrs to connect pipe.
In our case virtualbox provides own access rules of own pipe server side for "//./pipe/dos1" and the 'standard rules' for windows is 'single client only'.
1.2
One user told us abstract answer: "dup this to anything", so we can write a code to create two separated single_direction streams fcr/fcw.
The splitted fcr/fcw streams have no "r+" read/write interleave problems of original bi-directional fc stream.
Mode "a"/"w" is often not the same we need from "w" direction, so we use "r+" as "w" direction and to avoid r/w interleave problems just does not read from "r+".
1.3
The next improved version of initial example
example3
Now the example looks better, there are no more:
— fseek on pipe;
— fseek on every write to local file;
— fseek two times/per loop pass on every r/w interleave for pipe.
2.
There are some questions about <stdlib.h>.
About dup() behaviour.
Due to the guess about the dup() behaviour we can explain why:
— dup() can violate OS quotas for number of opened handlers;
— dup() can not "elevate up" RW access of system_handler created by open() (dup() can only more restrict the existing RW access created by open()).
It looks like process handler is abstract structure { system_handler data, local_handler data}.
open() creates both members in the abstract structure, but dup can change only local_handler field, copying the same system_handler field to new handler.
That means that from OS point of view, all duped handlers does the same access via the same system_handler, as if we several times called fread() for the same system_handler directly.
3.
Why we must not do fseek() on pipes?
Unrelated to stdio rules, we can not fseek() on devices which are incapable of seeking because there are no file pointer here, the devices are represented as pair {input_port, output_port}:
— there is no position of data inside each IO direction (not exist "input position" or "output position");
— there is no position of data between different IO directions ("input_port position" is not less, more or equal to "output position");
there are many devices that follow the restrictions.
And the "posix prog man" claims
— "for fseek() the value of the file offset on devices which are incapable of seeking is undefined".
that means stdio is ignoring {offset,whence} parameters of fseek() call on FILE which are incapable of seeking and fseek(fc, rand(), rand()) also will work in our example2 instead of assert( !fseek(fc,0,0) ).
The "posix prog man" claims
— "the behavior of fseek() on devices which are incapable of seeking is implementation-defined"
that means we must not call fseek() on devices which are incapable of seeking, fseek() must be used for directly purpose only, in order to set file pointer and not for any other jobs.
3.1
The question is ability to subst regular file instead of trivial pipe in the same program.
In order to work in program with regular file as if the file is a trivial pipe, we can restrict in the program fseek() usage to cases:
— sequential read file starting from begin of file fseek(fi,0,SEEK_SET);
— sequential write file by append after end of file fseek(fo,0,SEEK_END).
That means pipe should "defines up to complete" the prohibited fseek() by the two allowed fseek() calls with parameters '(0,SEEK_SET)/(0,SEEK_END)'.
The fseek() parameters have no sense for pipe but provide compatibility of pipe with regular file (SEEK_SET refer to read pipe, SEEK_END refer to write pipe).
sure, the purpose of stdio is to avoid us from dig the stream details and guess stream attr combinations, i have never looked so closely before.
3.2
We can imagine stdio based program, that does random access to regular file by fseek(offset), fread(size)/fwrite(size). Later we decided to feed the program by pipe instead of regular file, using external software created "gathered" pipe with "chuncks" of the same (offset,size).
In the case we need stdio behaviour is ignoring {offset, whence} parameters of fseek() call on pipe.
But in real the similar "gathered" pipe is just improper use of pipe, the programming technic produces unreliable code and leads to runtime errors, just believe that runtime errors is one of the most unpleasure thing you want to get.
Improper in comparison with "mapping regular file to pipe" that is implemented by only two special cases of fseek(fi,0,SEEK_SET)/fseek(fi,0,SEEK_END) calls allowed for pipe.
4.
why we need fseek() for r/w interleave in buffered "r+" mode?
FILE provides single buffer to IO and read/write data from downlayer handler by "chunks" into the buffer (reducing IO system calls).
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
but when in "r+" mode we change IO direction (read->write or write->read) we need flush dirty write buffer and drop (invalidate by LRU rules) obsolete read buffer.
But flush only is not enough, most time pointer in FILE and pointer in downlayer handler are not the same, we need fseek to change IO direction.
consider
"on devices which are incapable of seeking" in buffered "r+" mode just split pipe by fdup_r() or disable buffering by setvbuf(FILE, 0, _IONBF, 0).
5.
As result.
Once time earlier we have claimed "posix provides reliable way to access streams", but now we see this was not really true, there are some issues here.
And i have never seen the same info about dup() and 'r+' mode in "prog references".
Часть вторая
************
Let's continue to explore <stdio.h> stream access.
The original program did not work better because only a few errors returned by system calls was checked by assert() in the program.
If we check more returns we got more error messages, we got error messages on every wrong fputc/fgetc, that means stdio knows about the errors, but only we do not.
int
//fgetc emits pipe error if there was no fseek
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
...
//fputc emits pipe error if there was no fseek
assert( fputc(tx_ch,fc) != EOF );
...The improved version of initial example,
example2
enum{ PR_SYN= 0x16U };
fc= fopen("//./pipe/dos1","rb+"); assert(fc);
//unbuffered is always required for selected AUX port protocol "per char" exchange
assert( !setvbuf(fc, 0, _IONBF, 0) );
fi= fopen("./isw","rb"); assert(fi);
fo= fopen("./osw","ab"); assert(fo);
//
for(;;){
//rx
assert( !fseek(fc,0,SEEK_SET) );
int
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
assert( !fseek(fc,0,SEEK_END) );
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fc) != EOF );
}We could collect the error messages by more asserts, but the messages could not help us to understand "what the error is", because when we tried to access //./pipe/* from client side we did assume:
— we must not fseek() on pipes (will should always get ESPIPE error on every fseek());
— we can not violate OS quotas for number of pipe client side opened handlers (virtualbox pipe server side declares the quotas).
But in real <stdio.h> life, we not only "can", we "must", we should do assume:
— we must always DO fseek() on pipes when interleave read/write requests (otherwise will always get fputc/fgetc error);
— we can violate OS quotas for "number of pipe client side opened handlers" by dup() to create "rb"+"ab" separated access handlers pair;
— we must know dup() can not "elevate up" RW access of handler created by open() (dup() can only more restrict the existing RW access created by open()).
It is ridiculous, the real <stdio.h> conditions are strictly opposit to our expectations, and we must find why.
1.1
The OS quotas means we can not do like this
fcr= fopen("//./pipe/dos1", "rb"); assert(fcr);
fcw= fopen("//./pipe/dos1", "ab"); assert(fcw); //fcw always errorDue to the OS quotas we must use "r+" mode to r/w by one bi-directional handler
fc= fopen("//./pipe/dos1", "rb+"); assert(fc);As we know, windows has no posix "named FIFOs", means windows kernel does not provide "standard pipe server side" to any number of pipe clients, who has permissions defined by the file system chmod/chown attrs to connect pipe.
In our case virtualbox provides own access rules of own pipe server side for "//./pipe/dos1" and the 'standard rules' for windows is 'single client only'.
1.2
One user told us abstract answer: "dup this to anything", so we can write a code to create two separated single_direction streams fcr/fcw.
The splitted fcr/fcw streams have no "r+" read/write interleave problems of original bi-directional fc stream.
Mode "a"/"w" is often not the same we need from "w" direction, so we use "r+" as "w" direction and to avoid r/w interleave problems just does not read from "r+".
{
//here regular job for 'rb+' access
//splitted FILE can be buffered
//this is program responsibility to undestand the r/w buffers are shared or separated
//and provide coherence of multiple r/w shared buffers
//assert( !setvbuf(fc, 0, _IONBF, 0) );
//must be created in 'r+' access (O_RDWR)
//the original "rb+" will be used for "w" direction (O_WRONLY)
//dup() also can not "elevate up" RW attr, can not create "w" direction (O_WRONLY) from "r" direction (O_RDONLY)
fcw= fopen("//./pipe/dos1", "rb+"); assert(fcw);
//here function to do the FILE split regular job for 'rb+' access
{
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
fcr= fdopen(dfcr, "rb"); assert(fcr);
//drop dfc, dfcr values
}
//we can declare function
//to do the FILE split regular job for 'rb+' access
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(FILE *const fcw);
}1.3
The next improved version of initial example
example3
//
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(
FILE *const fcw
){
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
//fcr
return fdopen(dfcr, "rb");
//drop dfc, dfcr values
}
//
{
enum{ PR_SYN= 0x16U };
//pipe
//open pipe in "r+"("w") mode fcw
fcw= fopen("//./pipe/dos1","rb+"); assert(fcw);
//return "r" mode dupped fcr from original "r+"("w") mode fcw
fcr= fdup_r(fcw); assert(fcr);
//unbuffered pipe is always required for our AUX port protocol "per char" exchange
assert( !setvbuf(fcr, 0, _IONBF, 0) );
assert( !setvbuf(fcw, 0, _IONBF, 0) );
//allow subst regular file instead of pipe
assert( !fseek(fcr,0,SEEK_SET) );
assert( !fseek(fcw,0,SEEK_END) );
//local files
fi= fopen("./isw","rb"); assert(fi);
//append + reopen to r+
fo= fopen("./osw","ab+"); assert(fo); fclose(fo);
fo= fopen("./osw","rb+"); assert(fo); assert( !fseek(fo,0,SEEK_END) );
//exchange loop
for(;;){
//rx
int
rx_ch= fgetc(fcr);
if(rx_ch == EOF){ assert( feof(fcr) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fcw) != EOF );
}}Now the example looks better, there are no more:
— fseek on pipe;
— fseek on every write to local file;
— fseek two times/per loop pass on every r/w interleave for pipe.
2.
There are some questions about <stdlib.h>.
About dup() behaviour.
Due to the guess about the dup() behaviour we can explain why:
— dup() can violate OS quotas for number of opened handlers;
— dup() can not "elevate up" RW access of system_handler created by open() (dup() can only more restrict the existing RW access created by open()).
It looks like process handler is abstract structure { system_handler data, local_handler data}.
open() creates both members in the abstract structure, but dup can change only local_handler field, copying the same system_handler field to new handler.
That means that from OS point of view, all duped handlers does the same access via the same system_handler, as if we several times called fread() for the same system_handler directly.
3.
Why we must not do fseek() on pipes?
Unrelated to stdio rules, we can not fseek() on devices which are incapable of seeking because there are no file pointer here, the devices are represented as pair {input_port, output_port}:
— there is no position of data inside each IO direction (not exist "input position" or "output position");
— there is no position of data between different IO directions ("input_port position" is not less, more or equal to "output position");
there are many devices that follow the restrictions.
And the "posix prog man" claims
— "for fseek() the value of the file offset on devices which are incapable of seeking is undefined".
that means stdio is ignoring {offset,whence} parameters of fseek() call on FILE which are incapable of seeking and fseek(fc, rand(), rand()) also will work in our example2 instead of assert( !fseek(fc,0,0) ).
The "posix prog man" claims
— "the behavior of fseek() on devices which are incapable of seeking is implementation-defined"
that means we must not call fseek() on devices which are incapable of seeking, fseek() must be used for directly purpose only, in order to set file pointer and not for any other jobs.
3.1
The question is ability to subst regular file instead of trivial pipe in the same program.
In order to work in program with regular file as if the file is a trivial pipe, we can restrict in the program fseek() usage to cases:
— sequential read file starting from begin of file fseek(fi,0,SEEK_SET);
— sequential write file by append after end of file fseek(fo,0,SEEK_END).
That means pipe should "defines up to complete" the prohibited fseek() by the two allowed fseek() calls with parameters '(0,SEEK_SET)/(0,SEEK_END)'.
The fseek() parameters have no sense for pipe but provide compatibility of pipe with regular file (SEEK_SET refer to read pipe, SEEK_END refer to write pipe).
sure, the purpose of stdio is to avoid us from dig the stream details and guess stream attr combinations, i have never looked so closely before.
3.2
We can imagine stdio based program, that does random access to regular file by fseek(offset), fread(size)/fwrite(size). Later we decided to feed the program by pipe instead of regular file, using external software created "gathered" pipe with "chuncks" of the same (offset,size).
In the case we need stdio behaviour is ignoring {offset, whence} parameters of fseek() call on pipe.
But in real the similar "gathered" pipe is just improper use of pipe, the programming technic produces unreliable code and leads to runtime errors, just believe that runtime errors is one of the most unpleasure thing you want to get.
Improper in comparison with "mapping regular file to pipe" that is implemented by only two special cases of fseek(fi,0,SEEK_SET)/fseek(fi,0,SEEK_END) calls allowed for pipe.
4.
why we need fseek() for r/w interleave in buffered "r+" mode?
FILE provides single buffer to IO and read/write data from downlayer handler by "chunks" into the buffer (reducing IO system calls).
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
but when in "r+" mode we change IO direction (read->write or write->read) we need flush dirty write buffer and drop (invalidate by LRU rules) obsolete read buffer.
But flush only is not enough, most time pointer in FILE and pointer in downlayer handler are not the same, we need fseek to change IO direction.
consider
//read
fseek(0,0)
seek (0,0)
//here FILE and downlayer handler both points to 0
fgetc
//read into buf
read(0,4K)
//here FILE points to buf[1]
//and downlayer handler points to file[4K]
//char was returned from FILE buf[0]
//write
fseek(100,0)
//drop input buf
//here previous input [1..4K) will be dropped
//!HERE "on devices which are incapable of seeking" we can not fseek
//all dropped data will be lost, we can not "push back" the read data into pipe
//input stream data will be damaged just because we try write after read
//"on devices which are incapable of seeking" just split pipe by `fdup_r()`
//or disable buffering by `setvbuf(FILE, 0, _IONBF, 0)`
//seek new write pos 100
seek (100,0)
//here FILE and downlayer handler both points to 100
fputc
//write into buf
//here FILE points to 101
//char was placed into FILE buf[0]
//here downlayer handler points to 100
//write(100,1) was not called"on devices which are incapable of seeking" in buffered "r+" mode just split pipe by fdup_r() or disable buffering by setvbuf(FILE, 0, _IONBF, 0).
5.
As result.
Once time earlier we have claimed "posix provides reliable way to access streams", but now we see this was not really true, there are some issues here.
And i have never seen the same info about dup() and 'r+' mode in "prog references".