ringBufU Files

 r i n g B u f U
=================

### content
+ overview
+ release-notes
+ tutorial
+ work flow
+ testing
+ stest03 - results of a multithreading test
+ mingw compiler + msys
+/+





### overview

ringBufU is an C implementation of a 
ring-buffer.

"U" at the end is synonym for C "union". 
On creation time (kind of object structure 
with a returned instance ptr) you could choose 
the element-type from dedicated types:
- char
- long                  (Not in 0.1.0)
- long long
- void*
-/-
The usage of the specific element-type is done by a union 
via ptr.

ringBufU is not fully supported yet, just some basic features. You could see it 
in source-code by the NYI sign (o.c. Not-Yet-Implemented).

Yet integrated features:
+ general mutex support (POSIX), could be disable or changed 
  to own mutex/semaphor usage for single thread or 
  multi thread usage. Could be switched to single-thread / 
  no-mutex-usage by compilation setting (only global setting).
+ Delete protection of ringBufU instance for dynamical 
  usage without security issue. Could also be switched off 
  for single thread usage.
+ C++ compilable.
+ Macro wrapping for near object usage and namespace 
  identification.
+ Instance Ptr check with dummy usage in invalid Ptr case; instead 
  of crashes (e.g. segmantation-faults) only a specific return code 
  of 0x42FE (decimal 17150) appears to the calling functions.
+ put/get function to push/pop one element to/from the buffer. 
  (putX/getX planned to put/get more than 1 element at once)
+ Disclosure functions: isEmpty, isFull, isBufFlag.
  (isBufferOverflow, isBufferUnderrun, getFillSize planned).
+ Buffer-Overflow/-Underrun protection by default (detection by 
  return-code).
+ Usual int return-code with 0-is-no-error usage and not 0 
  is a specific error code for fast failure identification.
+ Fatal-Exit with code-info on unrealistic conditions which 
  usually occur only in corruption cases. E.g. if pthread_mutex 
  functions failed (error-check not useful in most cases).
+ Element-Types for Ring-Buffering could be
  - char
  - long long 
  - void*  (any ptr)
  - long                        - only in ringBufU versions >= V0.2
  -/-
  (Stack-buffering for all types NYI)
+ Scalable safe level (with variable RINGBUFU_FASTACC_NO >0, V>=0.2.1); less 
  safe configuration results usually in more speed of ringBufU APIs calls.

+
-------------------------------------------------------------------------------





### release-notes


* 0.1.0 - first release
  Just basic Ring-Buffer features for (char, long long, void*).

* 0.2.0 - second release
  Add   RINGBUFU_LOCKTYPE 21 for _WIN32
  Add   RINGBUFU_LOCKTYPE  2 for generic mutex in C++11 (~ ringbufuMutex.*)
  Chg   more comments
  Impl  fn getFillSize()
  Impl  fn getFillSizePromille()
  Impl  fn isBufferOverflow()
  Impl  fn isBufferUnderrun()
  Impl  fn isBufFlag()
  Add   some debug-functions (DCFn_..)
  Add   CPP-mutex support ~define RINGBUFU_USE_CPPMUTEX_ (make: CPPMUTEX!=1)
  Chg   stest01 could be used with pthreads(..) or CPP-mutex/threads (~CPPMUTEX)
  Add   stest03 : a multithreading test with 18 putter threads (in C+CPP)
  Add   tutor : including tutor#1 & tutor#2 (~ maintutor.c)
  Chg   Make a buffer max. size marging (MAX_UNITS_PER_BUF)
  Chg   do not use underscore at the begin of a var.name (_xyz -> xyz_)
  Add   support for RingBuffer-type for 'long' variables (RingLong_RBY)
  Add   stest02 + stest03 support for _WIN32
  Impl  VS support for stest02 & stest03 (subfolder VSprj/ringbufu/)

* 0.2.1 - release with optional acceleration
  Add   Test-features for automatic testing of specific configs (shell scripts)
  Chg   Enhanced C++ support
  Add   More comments and description of features/functions
  Impl  Different check stages for faster access (RINGBUFU_FASTACC_NO, >0)
  Add   RINGBUFU_VERSION for version info about current RingBufU
  Add   Global "RINGBUFU_*FailureCode"s, defs for generic failure in ringBufU
  Impl  Life time control for ensure valid ptrs (threads, RINGBUFU_FASTACC_NO>1)
  Add   Inquiry fn to get info about current config (ringbufu_getConfigString)
  Impl  Example inline fn to use unit directly (ringbufu_getLong)
  Chg   make: using predefined MAKE_CMD variable (usual == make)
  Add   options: general option checker "testargchk.c" for --help and more
  Impl  make: some make->C definitions (like RINGBUFU_FASTACC_NO, name 1:1)
  Add   make: target specific cleans (like clean_stest03 for stest03 target)
  Chg   make: using predefined MYCC variable (usual == gcc)
  Add   make: OS _WIN32 compiler mingw support (including msys for scr.testing)
  Add   make: variable MYOSTYPEID willB a define for C/CPP too (makeosdefs.mi)
  Chg   make: Define "__LINUX__" to "_LINUX___" (avoid system def doubling)
  Add   make: Define "_MACOS___" (ifdef === MYOSTYPE 1013)
  Add   RingBufU obj new API fn getBufferInfo() {buffer-type yet @1st only}
  Chg   Fix some issues
  Add   Support --fast and --config Option for some test applications
  Impl  ributest: an API check program (near unittest, ringbufu_test.c)
  Add   stest03: integrate alternative use of CPP std::queue (--useC++Q)
  Impl  stest03: use of multiple test instances (instead of one as before)
  Chg   stest03: optional change parameters of test (e.g. --loops X)

+
-------------------------------------------------------------------------------





### tutorial

The maintutor.c file could help you to understand the functionality of 
the usage of ringBufU instances.

There is at least one example, choose the correct number (tutor#) by setting 
the define "RINGBUFU_TUTOR_TESTNO" and enter 
> make tutor
<, then you can start this example by entering 
> ./tutor
<.
But please read the information above this example at 1st to know what will 
happen.

Every example has a main function and a specific init-function 
(tutor-number at the end of the init function name).

For more information pls. read the readme section in maintutor.c !

To see detailed information about a specific tutor-lesson you could search 
for the "RINGBUFU_TUTOR_TESTNO" (#elif...) and below this the informational 
text could be found.

* List:
tutor#| easy/mid/hard | brief    
---   | ------        | ---------------------------------------------------------------------------
001   | mid           | use 2 different buffers and move a text through
002   | easy          | put a entered line into a buffer, than print out char-by-char
---   | ------        | ---------------------------------------------------------------------------

+
-------------------------------------------------------------------------------
  




### work flow

1. initialisation
one-time prepare of ringBufU for secur usage.
>>
  ringbufu_init()
<<
Usual needed to call it one time before usage of RingBufU features. 
(needed only if RINGBUFU_INITEXITCALL_REQUIRED_ is defined 
or RINGBUFU_VERSION is not defined (?, may be not necessary)
or RINGBUFU_VERSION < 0x00000201 (0.2.1). Hence use it always if you are not 
sure.
).
--

2. creation of one buffer instance
Create the ring-buffer via create function.
>>
  Buffer_Instance_Ptr= ringbufu_createBuffer( BufferType, NULL,
                                              BufSizeInElements, 0 );
<<
{pBufbeg/arg2 NYI/NY-used, uMode/arg4 NYI}
Needed for every buffer before usage.
If Ptr return-code is not NULL, the buffer is prepared for usage in general.
Now some settings could be done, if needed. (e.g. disable lock for single 
thread usage).
Every access to this buffer-instance has to be done via this Ptr!
--

3. Buffer push access - put one element
Fill the buffer with elements, here for instance for a long-long buffer.
(BufferType = RingLLong_RBY)
>>
  DECL_4RINGBUFU_;                      // needed to use RINGBUFU_() macro
  tunRingBufU_Unit  uVar;               // for setup the specific element (cTrans,lTrans,llTrans,pTrans{for Ptr})
  int               iRC;                // to check the result of a macro call
  ..
  uVar.llTrans= long_long_variable_for_buffer_to_transfer; // input element (into buffer)
  iRC= RINGBUFU_( put, Ptr, &uVar );    // Ptr is the Buffer_Instance_Ptr, gotten via create-fn
  if( iRC) error-handling.. return..    // iRC==1: buffer overflow (other: see source)
<<
-
overflow: buffer is full, put-function is not able to save the given element.
Could be avoided by using the isFull function before (
>>
  if( RINGBUFU_( isFull, Ptr, 0)==0 )  put is ok
<<
but for locked access and multiple putter threads a better way is 
to check the return-code (iRC in example).
--

4. Buffer pop access - get one element
Get elements from the buffer.
(BufferType = RingLLong_RBY)
>>
  DECL_4RINGBUFU_;                      // needed to use RINGBUFU_() macro
  tunRingBufU_Unit  uVar;               // output element (from buffer)
  int               iRC;
  ..
  iRC= RINGBUFU_( get, Ptr, &uVar );
  if( iRC) error-handling.. return      // iRC==1: buffer underrun (other: see source)
  ...                                   // do something with uVar.llTrans, a long long var
<<
-
underrun: buffer is empty, get-function does not deliver an element yet.
Could be avoided by using the isEmpty function before (
>>
  if( RINGBUFU_( isEmpty, Ptr)==0 )  get is ok
<<
but checking return-code of the get-function may be sufficient.
--

5. Optionally delete buffer instance
Only for dynamically created and deleted buffers!
Or before program exit (not mandatory afai-think).
>>
  int  iRC;
  ..
  iRC= Ptr->destroy( &Ptr );
  if( iRC) error-handling.. return      // iRC==1: no valid buffer instance (other: see source)
  else ...                              // Ptr is set to NULL, 
                                        // buffer instance from previous Ptr is not 
                                        // available any longer 
<<
--

6. Optionally de-init ringBufU
Before exit or restart of a program. Before this de-init is possible, 
you have to delete/destroy every created buffer !
>>
  ringbufu_exit();
<<
--

+
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------





### testing

gen-info:
    > xyz   -> this means, enter xyz in a command-line window. short: in a CLI 
            which stands for Command-Line-Interface (a type of a 
            Computer user-interface,UI). {msys window for mingw}
            e.g. "> ./stest01" means: start test application "stest01".
            CLI will also be used in short for the CLI window/screen.
            Could be directly started (ubuntu for instance by holding keys 
            <Alt> + <Ctrl>, than press <T> {for Terminal}. more ways possible, 
            e.g. remote via ssh).
            This is now only described for linux/POSIX systems. There is just 
            small integration and test done for windows, sry (but a VS-project 
            exist for some tests and below a mingw usage info). 
            Test-Execution may take a lot of time; be patient. Although, if 
            output is not redirected, the time to display a new line or show 
            some changes in the CLI should not be more than 20 minutes.
            -
    <-> blabla
            -> this means, blabla is the expected result of the test before.
            "ok" means, the command returns with 0 exit code (
            in linux checkable by entering > echo $? <; in windows 
            by entering > echo %ERRORLEVEL% <).
            -
    i:      -> this means, behind i: is only some additional info which may 
            help in understanding.
            -
    preset  -> before testing at all use 
            > make clean
            <. 
            Just before starting the tests do this. It's already part 
            of release-tests.
            -
            -> before use an auto-script (autotest-C.sh, all-autotest.sh)
            DO: install valgrind (MAC: leaks)
            E.g. for ubuntu and derivates may be
            > sudo apt-get install valgrind
            < 
            works. (before number 2 of release tests , see below).
            Alternative to install valgrind, you could set "OPT_MEMCHK" 
            not to YES (use NO please) in autotest-C.sh!
            -
    Hint:   -> "BUILD_TYPE=RELEASE" is the current default for ringBufU.
            You could insert "BUILD_TYPE=DEBUG" before any make, which 
            enables debug-code.
            -
    --

    
ringBufU-test-info:
    ++  maintest  ++
    subj
        a small enhancable test programm <-> NOT YET AVAILABLE FOR ringBufU.
    make
        %
    using
        %
    help
        %
    ---
    
    ++  stestXX  ++
    subj
        short-test programs, XX= 01..03.
    make
        make stestXX - (linux...)
        VS build VS-project stestXX - (windows VS)
    using
        ./stestXX [options] - (linux style)
        stestXX.exe [options] - (windows style)
    help
        (./stestXX --help, e.g. ./stest02 --help aOPT_MEMCHKfter build stest02 with 
         make stest02; specific for stest02 is to get a sys-info via 
         ./stest02 --sys-info;
         ./xyz --config  shows the current config of ringBufU compilation 
         - almost usable for every test-program.
        )
    ---
    
    ++  tutorXX  ++
    subj
        short-tutor programs, XX= 01..02.
    make
        see stestXX
    using
        see stestXX
    help
        (./tutorXX --help, may be option "--fast" is also available)
    ---
    
    ++  all-autotest.sh  ++
    subj
        a test-shell script which tests smth via the registered test-programs 
        (array fulltest_cmds_ar) automatically
        optional you could build specific executables like stest01.
        This scripts uses multivaricall.sh and this script uses 
        autotest-C.sh, which could also be used directly.
        E.g. > ./autotest-C.sh stest01 deb_Defl < builds and run only 
        stest01 with a specific build-variant (shortname: deb_Defl).
    make
        no argument:
        - build & run  _ALL_  registered test-programs (different configs).
        argument = test-program ID, like stest01:
        - build & run only one registered test-programs, with this ID.
    restrictions
        currently only for MACOS and linux (and may be msys/mingw64).
    using
        > ./all-autotest.sh [ tutor01 | stest01 | ... ]
    help
        no help.
        read script and other ~--help~ to know what is done.
        back-leveling (??) -> to get more info if e.g. .>./all-autotest.sh stest01 < 
        failed, you could get back step-by-step, 
        e.g. 1st >./autotest-C.sh stest01 deb_Defl < or 
        >make clean_stest01 && make [config-options] stest01 <.
        Something in [] is just to use something with options; see help, script, 
        documentation,...; all but not including the square brackets.
    ---

    
++  short tests before a new release (linux & MACOS)  ++

1.
    i: in folder ../ringBufU/code/
    i: This is just a pretest to check if a specific test-program is 
    i: buildable and executable.
    > make clean
    <-> ok
    > make stest03
    <-> ok
    > ./stest03
    <-> ok  
    i:(be patient, this takes a lot of time, depending on your PC-machine, 
       OS and whatever. use -e.g.- arg "--loops 1000" to fasten it up.
      )
    --
2.
    > ./all-autotest.sh
    <-> ok
    i: be patient, this takes much more lot of time. Build all test-programs 
    i: with different presets (e.g. DEBUG/RELEASE) and let them run, and when 
    i: some errors occurs the execution stops (e.g. if in BUILD_TYPE DEBUG some 
    i: Debug-Check detects a issue, it could be result in an exit() call..).
    i: Belonging to a specific machine you should get infos in 
    i: ../ringBufU/result/<xyz-machine-spec-folder>/, e.g.
    i: in "times-info.txt" time outputs of some calls are registered during 
    i: execution of "./all-autotest.sh".
    i:(you also could use a program separator, e.g.
       > ./all-autotest.sh ributest -> calls just the ributest program multiple 
       times (in ringbufu_test.c, a short unittest program).
       > ./autotest-C.sh ributest deb_All1  -> calls the ributest program 
       (build and call) with the specific predefined build-set "deb_All1".
       Currently this means that make vars BUILD_TYPE and RINGBUFU_FASTACC_NO 
       are preset {BUILD_TYPE=DEBUG RINGBUFU_FASTACC_NO=} and a memchk 
       with valgrind (linux only) is enabled. E.g. if you put >>
                    void* pMEMCHK= malloc(20);
       <<
       in the main-fn of "ringbufu_test.c", the valgrind should detect the 
       mem-leak and the test-program exits (linux only).
      )
    --
    --
6.
    > make clean
    <-> ok
    --
    --
---

+    
-------------------------------------------------------------------------------





### stest03
some timings (not very exactly)

timing values with newer support of multiple instances.
default: 10 instances, 18 threads, 200000 loops.
machine     OS          LOCKTYPE    time    time,C++Q
PC1         mingw+msys  2            63s      7,4s
PC1         win10,VS    21           51s      3,7s      (VS-cfg: Release-x64)
PC1         Ubuntu*2    2           108s     12,2s
PC1         Ubuntu*2*3  1            82s      %
MAC         MACOSX*4    2            61s      8,4s
MAC         MACOSX*4*3  1            55s    ( 8,1s = ringBufU,if ..FAST..=4, *5)
-
Why are times very much bigger then C++Q times?
This is belonging to the fact of default compilation of ringBufU; this has 
maximum safe code - a wrong pointer of a ringBufU C~object~ does not result in 
an crash. afaik you do not have this option if you're using default C++ 
std::queue and std::mutex (let me know of i'm wrong!).
On the other hand with max. acceleration by using pre-make-var 
RINGBUFU_FASTACC_NO to 4, you should have near C++ speed with ringBufU 
(without safety, of course, like in C++ std Q). And nativ mutex, like 
pthread_mutex, may have little advantage before std::mutex and C++ compilation.
--
older info (not multiple instances)
machine     OS          LOCKTYPE    time    time,C++Q
PC1         win10       21          70s     
PC1         mingw+msys  21          -
PC1         ubuntu      1           41s
PC1         ubuntu      2           55s
MAC         MACOSX*1    1           29s
MAC         MACOSX*1    2           30s
---
*1: V=10.14.5
*2: V=20.04.6 LTS
*3: stest03C (use only C and C compiler)
*4: V=12.6.2
*5: 45s - config like *3, but with RINGBUFU_FASTACC_NO=1 as pre make variable
-------------------------------------------------------------------------------





### mingw compiler + msys

The mingw compilation (OS: Windows 10) and tests based on the following sets:
    -
    - from https://winlibs.com:
    Win64: 7-Zip archive* | Zip archive   -   without LLVM/Clang/LLD/LLDB: 7-Zip archive* | Zip archive
    {GCC 13.1.0 (with POSIX threads) + LLVM/Clang/LLD/LLDB 16.0.5 + MinGW-w64 11.0.0 (UCRT) - release 5   (LATEST)}
    -
    - msys version "MINGW64_NT-10.0-19045"
    -
    - dowmloads and infos from "http://www.cppbuch.de/downloads.html"
    -

Create the folder c:\cppbuchtools.
Create a link to this folder with a batch file "C:\cppbuchtools\mingwkonsole.bat"
(target/destination, DE:Ziel)
mingwkonsole.bat: {{
    @echo off
    if not exist "%~dp0bin\mkdirp.exe" goto FEHLER
    set PATH=%~dp0bin;C:\mingw64\bin;%PATH%;%~dp0sfml\lib
    cmd /k 
    goto :eof

    :FEHLER
    cmd /t:Cf /k echo FEHLER: Die Datei muss aus C:\cppbuchtools gestartet werden. Es kann auch eine Verknuepfung auf dem Desktop angelegt werden.
    exit
}}
(found sfml, boost, and sqlite in this folder).
-
copy "GNU coreutils online help: <https://www.gnu.org/software/coreutils/>" into 
sub-folder bin/
    
Install mingw-64 into folder: c:\mingw64

Install msys-64 into folder: C:\msys64

Open the folder by click on link (usually on desktop)
-> you'll be in the folder c:\cppbuchtools\.

Enter 
> C:\msys64\msys2_shell.cmd -mingw64 -where d:\xyz\workspace
< 
in this folder, when your base directory is d:\xyz\workspace; if not use 
another folder as argument after "-where".

Now you got a console like in linux.
You'll have the folder /home/<win-user-name>/ in this console.
The -where folder with windows drive i'll found in msys dir:
/d/xyz/workspace/ringBufU/code  (/d/ is for D:\ in windows afai-assume)
(This msys home-folder is in windows the folder 
 C:\msys64\home\<win-user-name>, e.g. if your 
 windows login name is      paffmaami , 
 the windows-folder is      C:\msys64\home\paffmaami\ ,
 and the msys folder is     /home/paffmaami/ .
 Both folders are the same, one you could access in windows (explorer..), and 
 the other in the msys console-window, which opens by calling the 
 "C:\msys64\msys2_shell.cmd" script.
)
I'll put something for convinience (4me) into the home folder, file .bashrc.
.bashrc: >>
    ...
    # adds by uws
    alias l='ls -al '
    # add drive to mingw bin (correction of path when using a separate folder c:\mingw64 - other config may possible)
    PATH=/c${PATH}
    export MAKE_CMD=C:/mingw64/bin/mingw32-make.exe
    alias make='${MAKE_CMD} '
<<

E.g. if you put ringBufU into d:\xyz\workspace\ringBufU\ you could use 
almost everthing like you have in linux.
BUT: you need a C++ compiler e.g. to use pthreads (?? i'd not found another 
method).
make be to build stest03 
enter 
> cd /d/xyz/workspace/ringBufU/code
> make stest03
-
After positive compilation you could start the test-program as usual in 
linux/MACOS console, e.g.:
> ./stest03 --instances 2 --useC++Q
or
> ./stest03 --instances 2 
or
> make RINGBUFU_FASTACC_NO=4 BUILD_TYPE=RELEASE stest03
> ./stest03 --instances 2 
-
Also the build-&-test scripts 
    autotest-C.sh  ,
    multivaricall.sh  and
    all-autotest.sh  
should be usable (at least i could not use tutor02 with automatic 
running via e.g. > ./all-autotest.sh tutor02 <.

+
-------------------------------------------------------------------------------