Я тут быстро репост сделаю методом "copy&paste" (не буду переводить много буков), а то люди в месте исходного размещения беспокоятся (так некрасиво улыбаются в мой адрес, что это приводит к юридическим последствиям (ограничения доступа и предоставления информации)) и как бы отвергают оплату за потребленные услуги. Они решили платить за все сами. Что же, так тому и быть (приятно больше не чувствовать себя должным)!
В кратце проблема в том, что есть VirtualBox отображающий гостевые UART 8250 порты на Windows хосте как "//./pipe" и VirtualBox работает как сервер для "//./pipe". На клиентской стороне такие порты могут быть открыты через такой простой интерфейс как <stdio> (что очень привлекательно).
Однако реально пользователи не могут обмениваться данными с такими гостевыми портами, потому что <stdio> у них не работает как надо, например потому что сервер VirtualBox устанавливает ограничения на количество подключений к "//./pipe". Как это исправить?
Оказывается <stdio> обладает рядом особенностей, которые пользователь в 99% случаев обычно никогда не встречает, но вот pipe требует особенной работы с <stdio>, и эти отличия не описаны ни в референсе для <stdio>, ни в книжках по posix. Я не видел. (подсказка: для программиста это интересная информация).
************
Часть первая
************
Assume you map Virtualbox guest OS COM port to Windows host pipe (\\.\pipe\*) and Virtualbox is server side of pipe.
It is possible to read/write Windows host pipe from client side (from Windows program) by <stdio>, i.e. access the pipe (and the guest COM port) as regular stream (via FILE, fopen, fgetc, fputc, etc).
1. The client side pipe behaviour details can depend on conventions offered by Virtualbox server side pipe implementation.
If you are not sure about the server side pipe considerations, you can find unexpected troubles while transfer data, among them:
— client side pipe terminates to receive rx data (windows program begin to read 0x00);
//the following code will not help to 'read after write'
int
//begin to read 0x00 here
rx_ch= fgetc(fc);
if(rx_ch == EOF)break;
— guest machine engine can hang guest OS window during COM port exchange via pipe (probably by pipe queue overflow);
— etc.
2.
By tests it was found, that there is the way to read/write client side pipe in half-duplex mode via single FILE handler opened for read/write by:
— for mingw pipe win directory is visible as "//./pipe"
— fopen pipe in "rb+" mode;
— mandatory calls fseek (with correct offset) between any read and write pipe operation.
Though the "fseek before append" is always mandatory for "r+" mode FILE, the fseek does nothing visible in pipe, but sets internal pipe data into correct state to carry out read and write pipe operation.
3.
example
enum{ PR_SYN= 0x16U };
fc= fopen("//./pipe/dos1","rb+"); assert(fc);
fi= fopen("./isw","rb"); assert(fi);
fo= fopen("./osw","ab"); assert(fo);
//
long
fc_fpos_rx= 0;
for(;;){
//rx
assert( !fseek(fc,fc_fpos_rx,SEEK_SET) );
int
rx_ch= fgetc(fc);
if(rx_ch == EOF)break;
fc_fpos_rx= ftell(fc); assert( fc_fpos_rx != -1L );
if(rx_ch != PR_SYN)fputc(rx_ch,fo);
//tx
assert( !fseek(fc,0,SEEK_END) );
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF)tx_ch= PR_SYN;
fputc(tx_ch,fc);
}
на фото: вот так работает такая программа
************
Часть вторая
************
Let's continue to explore <stdio.h> stream access.
The original program did not work better because only a few errors returned by system calls was checked by assert() in the program.
If we check more returns we got more error messages, we got error messages on every wrong fputc/fgetc, that means stdio knows about the errors, but only we do not.
int
//fgetc emits pipe error if there was no fseek
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
...
//fputc emits pipe error if there was no fseek
assert( fputc(tx_ch,fc) != EOF );
...
The improved version of initial example,
example2
enum{ PR_SYN= 0x16U };
fc= fopen("//./pipe/dos1","rb+"); assert(fc);
//unbuffered is always required for selected AUX port protocol "per char" exchange
assert( !setvbuf(fc, 0, _IONBF, 0) );
fi= fopen("./isw","rb"); assert(fi);
fo= fopen("./osw","ab"); assert(fo);
//
for(;;){
//rx
assert( !fseek(fc,0,SEEK_SET) );
int
rx_ch= fgetc(fc);
if(rx_ch == EOF){ assert( feof(fc) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
assert( !fseek(fc,0,SEEK_END) );
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fc) != EOF );
}
We could collect the error messages by more asserts, but the messages could not help us to understand "what the error is", because when we tried to access //./pipe/* from client side we did assume:
— we must not fseek() on pipes (will should always get ESPIPE error on every fseek());
— we can not violate OS quotas for number of pipe client side opened handlers (virtualbox pipe server side declares the quotas).
But in real <stdio.h> life, we not only "can", we "must", we should do assume:
— we must always DO fseek() on pipes when interleave read/write requests (otherwise will always get fputc/fgetc error);
— we can violate OS quotas for "number of pipe client side opened handlers" by dup() to create "rb"+"ab" separated access handlers pair;
— we must know dup() can not "elevate up" RW access of handler created by open() (dup() can only more restrict the existing RW access created by open()).
It is ridiculous, the real <stdio.h> conditions are strictly opposit to our expectations, and we must find why.
1.1
The OS quotas means we can not do like this
fcr= fopen("//./pipe/dos1", "rb"); assert(fcr);
fcw= fopen("//./pipe/dos1", "ab"); assert(fcw); //fcw always error
Due to the OS quotas we must use "r+" mode to r/w by one bi-directional handler
fc= fopen("//./pipe/dos1", "rb+"); assert(fc);
As we know, windows has no posix "named FIFOs", means windows kernel does not provide "standard pipe server side" to any number of pipe clients, who has permissions defined by the file system chmod/chown attrs to connect pipe.
In our case virtualbox provides own access rules of own pipe server side for "//./pipe/dos1" and the 'standard rules' for windows is 'single client only'.
1.2
One user told us abstract answer: "dup this to anything", so we can write a code to create two separated single_direction streams fcr/fcw.
The splitted fcr/fcw streams have no "r+" read/write interleave problems of original bi-directional fc stream.
Mode "a"/"w" is often not the same we need from "w" direction, so we use "r+" as "w" direction and to avoid r/w interleave problems just does not read from "r+".
{
//here regular job for 'rb+' access
//splitted FILE can be buffered
//this is program responsibility to undestand the r/w buffers are shared or separated
//and provide coherence of multiple r/w shared buffers
//assert( !setvbuf(fc, 0, _IONBF, 0) );
//must be created in 'r+' access (O_RDWR)
//the original "rb+" will be used for "w" direction (O_WRONLY)
//dup() also can not "elevate up" RW attr, can not create "w" direction (O_WRONLY) from "r" direction (O_RDONLY)
fcw= fopen("//./pipe/dos1", "rb+"); assert(fcw);
//here function to do the FILE split regular job for 'rb+' access
{
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
fcr= fdopen(dfcr, "rb"); assert(fcr);
//drop dfc, dfcr values
}
//we can declare function
//to do the FILE split regular job for 'rb+' access
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(FILE *const fcw);
}
1.3
The next improved version of initial example
example3
//
FILE
//return "r" mode dupped fcr from original "r+"("w") mode fcw
*fdup_r(
FILE *const fcw
){
int
dfcw, dfcr;
dfcw= fileno(fcw); assert(dfcw != 01L);
dfcr= dup(dfcw); assert(dfcr != 01L);
//fcr
return fdopen(dfcr, "rb");
//drop dfc, dfcr values
}
//
{
enum{ PR_SYN= 0x16U };
//pipe
//open pipe in "r+"("w") mode fcw
fcw= fopen("//./pipe/dos1","rb+"); assert(fcw);
//return "r" mode dupped fcr from original "r+"("w") mode fcw
fcr= fdup_r(fcw); assert(fcr);
//unbuffered pipe is always required for our AUX port protocol "per char" exchange
assert( !setvbuf(fcr, 0, _IONBF, 0) );
assert( !setvbuf(fcw, 0, _IONBF, 0) );
//allow subst regular file instead of pipe
assert( !fseek(fcr,0,SEEK_SET) );
assert( !fseek(fcw,0,SEEK_END) );
//local files
fi= fopen("./isw","rb"); assert(fi);
//append + reopen to r+
fo= fopen("./osw","ab+"); assert(fo); fclose(fo);
fo= fopen("./osw","rb+"); assert(fo); assert( !fseek(fo,0,SEEK_END) );
//exchange loop
for(;;){
//rx
int
rx_ch= fgetc(fcr);
if(rx_ch == EOF){ assert( feof(fcr) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
int
tx_ch= fi? fgetc(fi): PR_SYN;
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fcw) != EOF );
}}
Now the example looks better, there are no more:
— fseek on pipe;
— fseek on every write to local file;
— fseek two times/per loop pass on every r/w interleave for pipe.
2.
There are some questions about <stdlib.h>.
About dup() behaviour.
Due to the guess about the dup() behaviour we can explain why:
— dup() can violate OS quotas for number of opened handlers;
— dup() can not "elevate up" RW access of system_handler created by open() (dup() can only more restrict the existing RW access created by open()).
It looks like process handler is abstract structure { system_handler data, local_handler data}.
open() creates both members in the abstract structure, but dup can change only local_handler field, copying the same system_handler field to new handler.
That means that from OS point of view, all duped handlers does the same access via the same system_handler, as if we several times called fread() for the same system_handler directly.
3.
Why we must not do fseek() on pipes?
Unrelated to stdio rules, we can not fseek() on devices which are incapable of seeking because there are no file pointer here, the devices are represented as pair {input_port, output_port}:
— there is no position of data inside each IO direction (not exist "input position" or "output position");
— there is no position of data between different IO directions ("input_port position" is not less, more or equal to "output position");
there are many devices that follow the restrictions.
And the "posix prog man" claims
— "for fseek() the value of the file offset on devices which are incapable of seeking is undefined".
that means stdio is ignoring {offset,whence} parameters of fseek() call on FILE which are incapable of seeking and fseek(fc, rand(), rand()) also will work in our example2 instead of assert( !fseek(fc,0,0) ).
The "posix prog man" claims
— "the behavior of fseek() on devices which are incapable of seeking is implementation-defined"
that means we must not call fseek() on devices which are incapable of seeking, fseek() must be used for directly purpose only, in order to set file pointer and not for any other jobs.
3.1
The question is ability to subst regular file instead of trivial pipe in the same program.
In order to work in program with regular file as if the file is a trivial pipe, we can restrict in the program fseek() usage to cases:
— sequential read file starting from begin of file fseek(fi,0,SEEK_SET);
— sequential write file by append after end of file fseek(fo,0,SEEK_END).
That means pipe should "defines up to complete" the prohibited fseek() by the two allowed fseek() calls with parameters '(0,SEEK_SET)/(0,SEEK_END)'.
The fseek() parameters have no sense for pipe but provide compatibility of pipe with regular file (SEEK_SET refer to read pipe, SEEK_END refer to write pipe).
sure, the purpose of stdio is to avoid us from dig the stream details and guess stream attr combinations, i have never looked so closely before.
3.2
We can imagine stdio based program, that does random access to regular file by fseek(offset), fread(size)/fwrite(size). Later we decided to feed the program by pipe instead of regular file, using external software created "gathered" pipe with "chuncks" of the same (offset,size).
In the case we need stdio behaviour is ignoring {offset, whence} parameters of fseek() call on pipe.
But in real the similar "gathered" pipe is just improper use of pipe, the programming technic produces unreliable code and leads to runtime errors, just believe that runtime errors is one of the most unpleasure thing you want to get.
Improper in comparison with "mapping regular file to pipe" that is implemented by only two special cases of fseek(fi,0,SEEK_SET)/fseek(fi,0,SEEK_END) calls allowed for pipe.
4.
why we need fseek() for r/w interleave in buffered "r+" mode?
FILE provides single buffer to IO and read/write data from downlayer handler by "chunks" into the buffer (reducing IO system calls).
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
but when in "r+" mode we change IO direction (read->write or write->read) we need flush dirty write buffer and drop (invalidate by LRU rules) obsolete read buffer.
But flush only is not enough, most time pointer in FILE and pointer in downlayer handler are not the same, we need fseek to change IO direction.
consider
//read
fseek(0,0)
seek (0,0)
//here FILE and downlayer handler both points to 0
fgetc
//read into buf
read(0,4K)
//here FILE points to buf[1]
//and downlayer handler points to file[4K]
//char was returned from FILE buf[0]
//write
fseek(100,0)
//drop input buf
//here previous input [1..4K) will be dropped
//!HERE "on devices which are incapable of seeking" we can not fseek
//all dropped data will be lost, we can not "push back" the read data into pipe
//input stream data will be damaged just because we try write after read
//"on devices which are incapable of seeking" just split pipe by `fdup_r()`
//or disable buffering by `setvbuf(FILE, 0, _IONBF, 0)`
//seek new write pos 100
seek (100,0)
//here FILE and downlayer handler both points to 100
fputc
//write into buf
//here FILE points to 101
//char was placed into FILE buf[0]
//here downlayer handler points to 100
//write(100,1) was not called
"on devices which are incapable of seeking" in buffered "r+" mode just split pipe by fdup_r() or disable buffering by setvbuf(FILE, 0, _IONBF, 0).
5.
As result.
Once time earlier we have claimed "posix provides reliable way to access streams", but now we see this was not really true, there are some issues here.
And i have never seen the same info about dup() and 'r+' mode in "prog references".
************
Часть третья
************
Issues in exploring <stdio.h> stream access.
1.
bugfixes
When we does fseek() if the new file pointer points to data that is already in the buffer, nothing need to be done by IO system calls.
by stdio convention this is not true, fseek() will always reset FILE* buffer, only serial 'getc/putc' calls will skip IO system calls.
the convention is taken by stdio designers to simplify coherent FILE* access by the price of fseek() effectivity.
1.2
>"assert(??? != 01L)"
means "-01L"
1.3
>in function fdup_r(), "//drop dfcr"
never can "//drop dfcr" due to possible fdopen errors (now the function name is changed into fdup_pipe_r())
2.
Some summary info.
conceptually there are two parts of file handle: "system" and "per_process"
"system" has "per_process" references_list accessed by [process pid]
references_list item {
long owner_pid;
ulong num_of_the_process_dups;
};
references_list item will be removed if owner_pid terminated or no more `num_of_the_process_dups`
system handle will be closed if no more references_list "items"
system{
ulong lseek_pos;
references_list per_process[];
ubits system_io_restrictions;
}
process{
long system_fildes;
ubits per_process_io_restrictions;
}
2.1
there are several ways to create file handle that involves different parts of file handle:
IO call system per_process
open + + check access (chown,chmod) and create new both parts
fork - + dup "per_process" part for new proccess with the same io_restrictions
dup - + dup "per_process" part for current proccess with the same or more strict io_restrictions
dopen + + dup both parts with the same or more strict io_restrictions (is not supported by posix/stdlib)
close ? + close "per_process" part, close "system" part if no more "per_process" refs
2.2
we consider lseek_pos created as common (in "system part" of file handle) in order to simplify "read/write" calls to file handle interleaved by different processes without IPC locks usage, from first look there are no ways to get the same behaviour by private lseek_pos in "per_process" part.
coherent by process pair { lseek, read/write } requires IPC lock usage in IO access interleaved by different processes
dup does for file handle the similar as fork, but for current process
2.3
"fildes dopen(fildes, io_restriction)" works similar to "FILE* fdopen(fildes, io_restriction)"
the same as fdopen() or dup(), dopen() does not check access (chown,chmod) and uses existed system_io_restrictions created by open(). io_restriction is open() attr "O_RW/O_RO/O_WO".
dopen is not supported by posix/stdlib.
dopen() works with regular files and pipes.
for pipes file handle has no lseek() interface (lseek() returns ESPIPE), so for pipes ordinary dup() works very closely to dopen()
2.4
the our (not from lib) function fdup_pipe_r() is intended for bi_direction "r+" pipes only, to dup() from original bi_direction "r+"(O_RW) FILE* stream (created by open()/fopen()) new separated uni_direction "r"(O_RO) FILE* stream, in order to use:
— "r+" FILE* stream to uni_direction any buffered, pipe only write
— "r" FILE* stream to uni_direction any buffered, pipe only read
the function fdup_pipe_r() allows us to access guest UART 8250 port from host machine for virtualbox
3.
stdio FILE* dups
bi_direction "r+" streams
for bi_direction "r+" `FILE*` access, "pipe" is NOT base interface for any "regular file",
for interleaved r/w access in code is intended for "pipe" can not subst any "regular file",
the interleaving r/w for "regular file" requires `fseek()/ftell()`
for bi_direction "r+" `FILE*` access, "pipe" can not be buffered,
it is required to call `setvbuf(_IONBF)` for the "pipe",
the interleaving r/w for the "pipe" requires `push_back()` input `FILE*` buffer
for bi_direction "r+" `FILE*` access, "pipe" should be splitted by `fdup_pipe_r()`
into two separated uni_direction "r"/"w" `FILE*` access
to improve interleaved r/w access in code is intended for "pipe"
uni_direction "r"/"w" streams
for uni_direction "r"/"w" `FILE*` access, "pipe" IS base interface for SOME "regular file",
for uni_direction r/w access in code is intended for "pipe" can subst the "regular file",
in which the data is placed in real one-by-one as pipes does
so "pipe" interface should be "defined up to complete" by two fake calls:
`rewind()` the same as `fseek(0,SEEK_SET)`
`append()` the same as `fseek(0,SEEK_END)`
for uni_direction "r"/"w" `FILE*` access (including "r+" pipes splitted by `fdup_pipe_r()`)
"pipe" can be buffered by any `_IO?BF` type (by `setvbuf()` call)
so "r+" pipes splitted by `fdup_pipe_r()` in the two uni_direction "r"/"w" derived pipes
povide any buffered interleaved r/w access in code is intended for "pipe"
(by the two uni_direction "r"/"w" derived pipes working together)
it is prohibited to create code is intended for "pipe" in uni_direction "r"/"w" FILE* access, in which can subst any (without one-by-one placed data) "regular file" by fake fseek(rand(),SEEK_SET) calls in the code
if code needs `fseek(rand(),SEEK_SET)` calls, the code is not intended for pipes,
the code follows "regular file" interface (by `fseek()` requests),
but "regular file" is not base interface for "pipe".
attempt to subst "pipe" instead of "regular file" interface violates "type checking"
The next improved version of initial example
example4
//
FILE
//return "r" mode dupped fr from original "r+"("w") mode fw
*fdup_pipe_r(
FILE *const fw
){
int
dfw, dfr;
//assume "fileno" returns the same "fildes" value that is stored in the FILE*
//("fileno" does not make "dup")
dfw= fileno(fw); assert(dfw != -01L);
dfr= dup(dfw); assert(dfr != -01L);
//fr
FILE
*fr;
fr= fdopen(dfr, "rb"); if(!fr)close(dfr);
return fr;
//drop dfw, dfr values
}
//
{
enum{ PR_SYN= 0x16U };
//pipe
//open pipe in "r+"("w") mode fcw
fcw= fopen("//./pipe/dos1","rb+"); assert(fcw);
//return "r" mode dupped fcr from original "r+"("w") mode fcw
fcr= fdup_pipe_r(fcw); assert(fcr);
//unbuffered pipe is always required for our AUX port protocol "per char" exchange
//but for splitted pipes any buffering type "_IO?BF" could be set
assert( !setvbuf(fcr, 0, _IONBF, 0) );
assert( !setvbuf(fcw, 0, _IONBF, 0) );
//"r+" pipe in interleaved r/w access never allows subst regular file instead of pipe
//("rewind()/append()" calls can not help)
//assert( !fseek(fcr,0,SEEK_SET) );
//assert( !fseek(fcw,0,SEEK_END) );
//our AUX port protocol local files
fi= fopen("./isw","rb"); assert(fi);
//"a"(create if not exist) + reopen to "r+" + append()
fo= fopen("./osw","ab+"); assert(fo); fclose(fo);
fo= fopen("./osw","rb+"); assert(fo); assert( !fseek(fo,0,SEEK_END) );
//our AUX port protocol exchange loop
//terminates by ctrl+C
for(;;){
//rx
int
rx_ch= fgetc(fcr);
//"assert( feof(fcr) )" throw error if fgetc(fcr) returned EOF but !feof(fcr) (but better check errno)
//otherwise endless loop with intention
if(rx_ch == EOF){ assert( feof(fcr) ); break; }
if(rx_ch != PR_SYN)assert( fputc(rx_ch,fo) != EOF );
//tx
int
tx_ch= fi? fgetc(fi): PR_SYN;
//"assert( feof(fi) )" throw error if fgetc(fi) returned EOF but !feof(fi) (but better check errno)
//otherwise endless loop with intention
if(tx_ch == EOF){ assert( feof(fi) ); tx_ch= PR_SYN; }
assert( fputc(tx_ch,fcw) != EOF );
}}
Now the example looks better. guest UART 8250 port will be accessed by simple stdio read/write.
