José Luis Esteban
Version |
0.0 2015-04-03 |
Commit |
"f02a72b50e51c52cda7b69fa2277563311913a6a" |
Generated |
2016-03-27 23:53:04 CEST |
1. Introduction
1.1. About
1.1.1. Why
C++ is growing incredible fast and it will continue in next future. To get more complete library, variadic templates, lambdas, string-view, smart-pointers, move semantic, concurrency, files, initializer lists, type inference…
To learn, to practice new features from C++, to have a toolset to complete C++ library, to program in C++ with less "undefined behavior"
Once upon a time…
I wrote some rules and C++ tools to help colleagues with no experience in C++ to survive with it.
But later, I discover It was pleasant to work with these tools, safe, easy to use, low dependencies… I’m also very interested in making safe programs (no crashes)
This toolset compiles with modern C++ (and it will be), but is not fully idiomatic. Some of the tools has been added to C++ standard library (regular expressions, tuples, date time…), and there are new ways to solve most of the problems. It was time to play and learn with new C++ and these concepts
Let’s enjoy and learn new C++. Let’s have toolkit working with new C++. Let’s create your basic tools to develop from scratch. It’s a great excuse!!!
I loved my CountPtr implementation, my old tuples worked fine… Now both will be replaced by std implementation. Other tools, will be rewrited with modern C++.
I know It will be necessary to update frequently some parts of the library in order to keep updated to last versions of C++. It’s part of the game.
Previously I wrote the documentation with asciidoc, muse over emacs and doxys on some projects. Now I’m going to add doxygen (I’m not a fan of these kind of tools, I prefer something like asciidoc or emacs/muse, but it’s a standard).
1.1.2. Directives
I’m not focused on radical performance, I prefer:
-
Safety
-
I don’t like undefined behaviour
-
I don’t like crashes due to programming mistakes. C++ could be a dangerous language. It’s easy to have crashes, I want a library protecting and helping the programmer.
-
Message driven programming
-
If we have a failure processing a message, usually is better to inform about it and continue with next one.
-
KISS and YAGNI
-
Easy to use API (the implementation could be complex)
-
I will introduce features as soon I will require
-
No thousands of options, better few options, easy to use and less verbose (try to compose, please)
-
Make difficult to produce "write only code" using this library. The code using this library…
-
has to be easy to read
-
writting simplicity is less important
-
Small
-
This is about having basic features, not incredible possibilities. There are great C++ libraries for that.
-
Portability (win/linux)
-
zero dependency
-
C++ doesn’t have a standard dependency tool. It is going to be a small library, complex dependencies will be avoided
-
Easy to compile
Performance is desired but not over safety, easy api…
1.2. Safety
Important working on a team. |
C++ is not the safest programming language in the world. In fact, it’s easy to produce undefined behaviour working with C++.
undefined behavior is all arround you… |
Working in a team (generally different levels), it’s desirable to have safety in mind.
C++ is focused on performance and low level abstractions.
In C++ you can have undefined behaviour for several errors.
Most of these undefined behaviours ends with a program crash.
My aim is to reduce program crashes. I want my programs keep running even after an error.
I don’t try to transform C++ in Erlang or Elixir. And yes, I know that Rust is the tool which try to be efficient, close to metal and safe.
I enjoy learning and practicing Erlang/elixir. I learned several concepts with them. Rust looks quite interesting and promising, we will see what happens in the future, meanwhile, I’m learning and following it.
Crash and errors reasons and how to deal with them…
Crash | Managed with… |
---|---|
Non trapped exception |
message driven programming with main message loop trapping all exceptions |
Null pointer |
generate an exception is better to undefined behaviour (usually crash) |
Dangling pointers |
smart pointers and signal slot. Please RAII everywhere <br> No arithmetic pointers |
Memory leaks |
same as previous one |
Resource management |
memory is not the only resource. RAII everywhere |
Invalid iterators |
Exception is better than undefined behaviour (even for deterministic false positives, fail fast) |
Infinite loops |
reduce loops usage |
Consuming all resources |
_ |
Integer zero division |
throw an exception |
Concurrency and parallelism |
reduce or remove parallelism <br> do not share, message passing |
Float comparison |
don’t do it and request for help to the compiler |
Race conditions |
Share nothing, send messages, high level concurrency tools, or… reduce parallelism |
1.2.1. Let the compiler help you
I have next flags activated on gcc/g++
-std={cpp}14 -O0 -g -Werror -Wall -W -Wundef -Wpointer-arith -Wfloat-equal -fexceptions -Winit-self -Wconversion -Wclobbered -Wempty-body -Wignored-qualifiers -Wmissing-field-initializers -Wsign-compare -Wtype-limits -Wuninitialized -Wno-unused-result -Wnon-virtual-dtor -Wreorder -Woverloaded-virtual -Wsign-promo -Winit-self -Wignored-qualifiers -Wmissing-include-dirs -Wswitch-default -Wswitch-enum -Wshadow -Wcast-qual -Wwrite-strings -Wconversion -time
jle will also provide a base exception class with stack. You will have to fill the stack manually (this is C++)
1.3. Concurrency
Important working on a team. |
Concurrency is great. Why?
-
Several problems are easy to solve in a concurrent way
-
Avoid active waiting
-
Use all machine cores (better performance)
-
Avoid full program stop waiting for a task
I love concurrency and parallelism, but I love it with languages like Erlang and Elixir, designed to work great with this concept.
ADA and Rust, would be interesting candidates.
But Python, Ruby not due to GIL, GVL, to start with.
C, C++, Java, C#… aren’t good for concurrency. They lack of high level abstractions and they are not designed to avoid race conditions.
You could use different strategies to avoid concurrency problems, like resources locking ordering. All these kind of strategies, reduce the concurrency and the code continues being difficult to maintain.
You could have a great thread safe code working perfect. But some day, you could call a different function and your code, could not be thread safe anymore. This will be difficult to detect and very difficult to solve.
The majority of Chrome code is intended to be single-threaded, where this presents no problem. When in multi-threaded code, however, the right answer is usually to use a base::LazyInstance.
http://www.chromium.org/developers/coding-style/cpp-dos-and-donts
The right way to deal with concurrency is… "share nothing, message passing" (actor model and csp)
Therefore, threads are not a good idea. In Rust, could be an option because the compiler will forbid you to share things between threads.
1.3.1. Solve easily some problems
Computer is a state machine. Threads are for people who can’t program state machines
Message passing in an ansynchronous way, also generates new problems. Many times we need a synchronous communication. Erlang/Elixir solves it.
As Alan Cox said, you can develop state machines. In fact, all non trivial process, has to deal with states.
I will create an external DSL to write declarative finite state machines.
1.3.2. Avoid active waiting
For asynchronous task like reading a socket.
OK, do it, wait for asynchronous events on a dedicated thread.
You can even execute your code in a dedicated thread, but not simultaneously with other code of your own program.
Doing it, will be as easy as adding a line JLE_ASYNCHR
1.3.3. Using all machine cores
Do it with processes. You can communicate them with pipes, rabbitmq, RESTful…
This way, you can use all cores and even all available machines.
Concurrency with processes… is share nothing communicate with messages. The right way.
I will add support for RESTfull, rabbitmq, execute process and communicate with pipes.
1.3.4. Avoid program stop waiting for a task
As before, send it to a specific process configured to work with heavy and slow tasks.
1.4. Small example
#include <iostream>
#include "core/alarm.h"
#include "core/signal_slot.hpp"
#include "core/timer.h"
// Function to receive timer events
void test_timer(void)
{
std::cout << jle::chrono::now() << " on_timer in fucntion" << std::endl;
}
int main()
{
std::cout << jle::chrono::now() << " starting..." << std::endl;
// configure timer for function
JLE_TIMER_FUNCT(1s, test_timer);
// program stop after 10s
JLE_ONE_SHOOT_FUNCT(10s, [](){ std::cout << "CLOSING... "; jle::timer::stop_main_loop();});
jle::timer::start_main_loop();
}
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << al << std::endl;
}
1.5. Folders
Folder | Description |
---|---|
src |
source code |
src/core |
Basic tools (signal-slot, strings…) |
src/net |
net source |
src/xxx |
pending |
bin |
generated bins |
doc |
generated doc |
test |
source for unit testing |
examples |
ex source |
data |
general data |
1.6. Compile
Next commands are provided
make make help make libs make doc make test make compile_test make compile_examples
1.7. Roadmap
I don’t plan to use it daily. I will write it simultaneously with… web applications with dart, polymer, enjoining Elixir, learning from Go and Rust, following Scala…
In any case, I plan to follow next order (more or less)…
-
(done) smart_pointer
-
(done) just a safe wrapper over stl, but safer
-
-
(done) signal_slot
-
(90%) date_time
-
(done) string tools
-
(done) exception type with stack
-
(done) double safe comparison
-
(done) safe containers
-
(done) nullable type, it is a wrapper from std::experimental::optional
-
(done) tuples ostream <<
-
(50%) Http REST support (pending routes and client)
-
(done) integer div 0 protection
-
(90%) Message driven programming oriented: synchr, timers
-
(done) parallelism control helper
-
(done) LL(n) parser
-
(done) qt gui for LL(n)
-
LL(n) parser documentation
-
(done) IDL class generation base and example
-
IDL class generation
-
stream
-
yaml
-
json
-
bson
-
less operator
-
equal operator
-
…
-
-
IDL documentation
-
.ini and .cfg parsing files
-
ashared_ptr. Destroy it ansynchronous way to avoid deleting when using it
-
IDL fsm generation
-
async signals
-
soft-realtime facilities
-
factory template
-
…
1.8. Todo
-
…
2. STATS
_ | lines | files |
---|---|---|
/ |
32263 |
91 |
src/ |
12282 |
38 |
src/core/ |
9363 |
35 |
src/net/ |
460 |
2 |
examples/ |
15064 |
29 |
test/ |
1786 |
10 |
tools/ |
3010 |
13 |
extern | lines | files |
---|---|---|
src/net/ |
10104 |
2 |
3. lib_core
3.1. shared_ptr
Safe std::shared_ptr wrapper
Same interface as std::shared_ptr but, with no undefined behaviour
#include <iostream>
#include "core/shared_ptr.hpp"
struct Foo {
Foo() { std::cout << "Foo...\n"; }
~Foo() { std::cout << "~Foo...\n"; }
};
struct D {
void operator()(Foo* p) const {
std::cout << "Call delete for Foo object...\n";
delete p;
}
};
int main()
{
{
std::cout << "constructor with no managed object\n";
jle::shared_ptr<Foo> sh1;
}
{
std::cout << "constructor with object\n";
jle::shared_ptr<Foo> sh2(new Foo);
jle::shared_ptr<Foo> sh3(sh2);
std::cout << sh2.use_count() << '\n';
std::cout << sh3.use_count() << '\n';
}
{
std::cout << "constructor with object and deleter\n";
jle::shared_ptr<Foo> sh4(new Foo, D());
}
{
std::cout << "valid pointer" << std::endl;
auto ptr = jle::make_shared<int>(22);
std::cout << "expired " << int(ptr.expired()) << std::endl;
}
{
std::cout << "invalid pointer" << std::endl;
auto ptr = jle::make_shared<int>(22);
ptr.reset();
std::cout << "expired " << int(ptr.expired()) << std::endl;
}
{
std::cout << "invalid pointer" << std::endl;
jle::shared_ptr<int> ptr;
std::cout << "expired " << int(ptr.expired()) << std::endl;
}
{
jle::shared_ptr<int> foo = jle::make_shared<int> (10);
// same as:
jle::shared_ptr<int> foo2 (new int(10));
auto bar = jle::make_shared<int> (20);
auto baz = jle::make_shared<std::pair<int,int>> (30,40);
std::cout << "*foo: " << *foo << '\n';
std::cout << "*bar: " << *bar << '\n';
std::cout << "*baz: " << baz->first << ' ' << baz->second << '\n';
}
{
std::cout << "invalid pointer" << std::endl;
auto ptr = jle::make_shared<int>(22);
ptr = {};
std::cout << "expired " << int(ptr.expired()) << std::endl;
}
}
3.2. weak_ptr
Safe std::weak_ptr wrapper
Same interface as std::weak_ptr but, with no undefined behaviour
#include <iostream>
#include "core/shared_ptr.hpp"
jle::weak_ptr<int> gw;
void f()
{
jle::shared_ptr<int> spt = gw.lock(); // Has to be copied into a shared_ptr before usage
if (spt.expired()==false) {
std::cout << *spt << "\n";
}
else {
std::cout << "gw is expired\n";
}
}
int main()
{
{
auto sp = jle::make_shared<int>(42);
gw = sp;
f();
}
f();
}
3.3. Test
Some macros and functions to help with test core/test.h
briefly…
Macro | Description |
---|---|
JLE_TEST_FILE |
at the begining of execution file |
JLE_TEST_REPORT |
at the end of execution file |
JLE_TEST_INIT |
at the beggining of a test (generally a function) |
JLE_TEST_ASSERT |
check truth and write a dot |
JLE_TEST_ASSERT_NO_DOT |
check truth without writting a dot |
JLE_TEST_EXCEPTION |
check expression throws an exception |
#include <iostream>
#include "core/shared_ptr.hpp"
#include "core/misc.h"
#include "core/test.h"
struct Foo {
Foo(int _i, int _j) : i(_i), j(_j) { }
~Foo() { }
int i;
int j;
};
void test_use_count()
{
JLE_TEST_INIT
{
jle::shared_ptr<Foo> p;
JLE_TEST_ASSERT(p.use_count() == 0);
jle::shared_ptr<Foo> p1(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
jle::shared_ptr<Foo> p2(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
JLE_TEST_ASSERT(p2.use_count() == 0);
jle::shared_ptr<Foo> p3(p2);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
JLE_TEST_ASSERT(p2.use_count() == 0);
JLE_TEST_ASSERT(p3.use_count() == 0);
}
{
jle::shared_ptr<Foo> p{new Foo{42, 26}};
JLE_TEST_ASSERT(p.use_count() == 1);
jle::shared_ptr<Foo> p1(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(p1.use_count() == 2);
jle::shared_ptr<Foo> p2(p);
JLE_TEST_ASSERT(p.use_count() == 3);
JLE_TEST_ASSERT(p1.use_count() == 3);
JLE_TEST_ASSERT(p2.use_count() == 3);
jle::shared_ptr<Foo> p3(p2);
JLE_TEST_ASSERT(p.use_count() == 4);
JLE_TEST_ASSERT(p1.use_count() == 4);
JLE_TEST_ASSERT(p2.use_count() == 4);
JLE_TEST_ASSERT(p3.use_count() == 4);
}
{
jle::shared_ptr<Foo> p= jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::shared_ptr<Foo> p1(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(p1.use_count() == 2);
jle::shared_ptr<Foo> p2(p);
JLE_TEST_ASSERT(p.use_count() == 3);
JLE_TEST_ASSERT(p1.use_count() == 3);
JLE_TEST_ASSERT(p2.use_count() == 3);
jle::shared_ptr<Foo> p3(p2);
JLE_TEST_ASSERT(p.use_count() == 4);
JLE_TEST_ASSERT(p1.use_count() == 4);
JLE_TEST_ASSERT(p2.use_count() == 4);
JLE_TEST_ASSERT(p3.use_count() == 4);
}
{
jle::shared_ptr<Foo> p= jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
{
jle::shared_ptr<Foo> p1(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(p1.use_count() == 2);
jle::shared_ptr<Foo> p2(p);
JLE_TEST_ASSERT(p.use_count() == 3);
JLE_TEST_ASSERT(p1.use_count() == 3);
JLE_TEST_ASSERT(p2.use_count() == 3);
}
jle::shared_ptr<Foo> p3(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(p3.use_count() == 2);
}
{
jle::shared_ptr<Foo> p;
JLE_TEST_ASSERT(p.use_count() == 0);
jle::shared_ptr<Foo> p1(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
jle::shared_ptr<Foo> p2 = jle::make_shared<Foo>(42, 42);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
JLE_TEST_ASSERT(p2.use_count() == 1);
jle::shared_ptr<Foo> p3(p2);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(p1.use_count() == 0);
JLE_TEST_ASSERT(p2.use_count() == 2);
JLE_TEST_ASSERT(p3.use_count() == 2);
}
}
void test_weak_ptr()
{
JLE_TEST_INIT
{
jle::shared_ptr<Foo> p;
JLE_TEST_ASSERT(p.use_count() == 0);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(w.expired() == true);
}
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
}
{
jle::weak_ptr<Foo> w;
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(p.use_count() == 1);
w=p;
JLE_TEST_ASSERT(w.expired() == false);
}
}
{
jle::weak_ptr<Foo> w;
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(p.use_count() == 1);
w=p;
JLE_TEST_ASSERT(w.expired() == false);
}
JLE_TEST_ASSERT(w.expired() == true);
}
}
void test_expired()
{
JLE_TEST_INIT
{
jle::weak_ptr<Foo> w;
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(p.use_count() == 1);
w=p;
JLE_TEST_ASSERT(w.expired() == false);
}
JLE_TEST_ASSERT(w.expired() == true);
}
}
void test_operators()
{
JLE_TEST_INIT
{
jle::shared_ptr<Foo> p;
p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
}
{
jle::shared_ptr<Foo> p {jle::make_shared<Foo>(42, 26)};
jle::shared_ptr<Foo> p1{p};
JLE_TEST_ASSERT(p.use_count() == 2);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(w.expired() == false);
}
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
JLE_TEST_ASSERT(w.lock()->i == 42);
JLE_TEST_ASSERT((*(w.lock())).j == 26);
JLE_TEST_ASSERT(w.lock().get()->i == 42);
JLE_TEST_ASSERT((*(w.lock()).get()).j == 26);
}
}
void test_exception_on_expired()
{
#define JLE_TEST_EXCEPTION(__EXPRESION__) \
{bool except = false; try { (void) (__EXPRESION__); } catch(...) { except=true; }; JLE_TEST_ASSERT(except);}
JLE_TEST_INIT
{
jle::shared_ptr<Foo> p;
JLE_TEST_ASSERT(p.use_count() == 0);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(w.expired() == true);
p.reset();
JLE_TEST_EXCEPTION(w.lock()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock())).j == 26);
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
{
jle::shared_ptr<Foo> p;
JLE_TEST_ASSERT(p.use_count() == 0);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(w.expired() == true);
p = jle::shared_ptr<Foo>{};
JLE_TEST_EXCEPTION(w.lock()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock())).j == 26);
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
{
jle::shared_ptr<Foo> p;
p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
p.reset();
JLE_TEST_EXCEPTION(w.lock()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock())).j == 26);
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
{
jle::shared_ptr<Foo> p {jle::make_shared<Foo>(42, 26)};
jle::shared_ptr<Foo> p1{p};
JLE_TEST_ASSERT(p.use_count() == 2);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 2);
JLE_TEST_ASSERT(w.expired() == false);
p.reset();
JLE_TEST_EXCEPTION(p->i == 42);
JLE_TEST_EXCEPTION((*p).j == 26);
JLE_TEST_ASSERT(p1.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
p1.reset();
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
{
jle::shared_ptr<Foo> p = jle::make_shared<Foo>(42, 26);
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
JLE_TEST_ASSERT(w.lock()->i == 42);
JLE_TEST_ASSERT((*(w.lock())).j == 26);
JLE_TEST_ASSERT(w.lock().get()->i == 42);
JLE_TEST_ASSERT((*(w.lock()).get()).j == 26);
p.reset();
JLE_TEST_EXCEPTION(w.lock()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock())).j == 26);
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
}
void test_make_shared_ptr()
{
}
void test_reset(void)
{
{
jle::shared_ptr<Foo> p {jle::make_shared<Foo>(42, 26)};
JLE_TEST_ASSERT(p.use_count() == 1);
jle::weak_ptr<Foo> w(p);
JLE_TEST_ASSERT(p.use_count() == 1);
JLE_TEST_ASSERT(w.expired() == false);
p.reset();
JLE_TEST_ASSERT(p.use_count() == 0);
JLE_TEST_ASSERT(w.expired() == true);
JLE_TEST_EXCEPTION(w.lock()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock())).j == 26);
JLE_TEST_EXCEPTION(w.lock().get()->i == 42);
JLE_TEST_EXCEPTION((*(w.lock()).get()).j == 26);
}
}
int main()
{
JLE_TEST_FILE
test_use_count();
test_weak_ptr();
test_expired();
test_operators();
test_exception_on_expired();
test_make_shared_ptr();
test_reset();
JLE_TEST_REPORT
}
3.4. String
3.4.1. Concatenation JLE_SS
Code like…
std::string s = "asdfds" + var1 + var2 + "\n";
is very dangerous and limited.
It’s better…
std::cout << "asdfds" << var1 << var2 << std::endl;
That is safer and powerful.
Please, never use +
to concat strings. My proposal is…
JLE_SS("asdfds" << var1 << var2 << std::endl);
JLE_SS will produce a ostringstream
with all it powers (and safer than +), therefore more expressive
3.4.2. Converting
s_try_sxxx converts an string to integer, double, duration, time point…
You can provide a value in case conversion fail. It will return a tuple with the converted value or the default and a boolean indicating if conversion was right
Another utilities are…
-
Trimming
-
Align
-
Splitting
#include <iostream>
#include <tuple>
#include "core/string.h"
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << "ouch...: " << al << std::endl;
}
int main()
{
{
auto result = jle::s_try_s2d("3.1415926535", 0.);
if(std::get<1>(result))
std::cout << std::get<0>(result) << std::endl;
}
{
auto result = jle::s_try_s2dur("3m 4s 132ms", 0s);
if(std::get<1>(result))
std::cout << std::get<0>(result) << std::endl;
}
}
3.5. signal_slot
An good event system is great in order to compose. signal_slot is a static event system (good performance and strict compiling type check)
I wrote one years before. I had to deal with optional parameters on templates. This implementation worked with 0 till 4 parameters. In order to add parameters, was necessary to modify the code. Very similar code. Yes, it was methaprogramming and it was copy/paste (disgusting)
Today, with modern C++, I wrote it with variadic templates. It has been a fantastic experience. Quite easy, and it works with several parameters
If composition with events is not enougth for you, signals are also fully RAII. When signal or receiver gets out of scope, signals are disconnected. You don’t have to take care of disconnection before exit (destructor), neither disconnection to avoid orphan live references (problem for counter ptrs and garbage collectors)
3.5.1. Connection syntax recomendations…
Connection | Syntax |
---|---|
signal → function |
signal.connect(on_function_name); |
signal → method |
JLE_CONNECT_METHOD(signal, instance, on_method); |
signal → method on this |
JLE_CONNECT_THIS(signal, on_method); |
signal → signal |
signal.connect(&signal_receiver); |
3.5.2. Performance
In my computer with no params, you can emit around 200.000 signals per millisecond…
Executing… @link /core/signal_slot_performance.cpp @endlink
--------------------------------------------------------------------- add int loop 1783293664 time: 0.000598493 calls/millisecond: 1.67086e+06 --------------------------------------------------------------------- signal -> function (emit) time: 0.00495689 calls/millisecond: 201740 --------------------------------------------------------------------- signal -> function (no emit) time: 0.00491906 calls/millisecond: 203291 --------------------------------------------------------------------- signal -> method (emit) time: 0.00461001 calls/millisecond: 216919 --------------------------------------------------------------------- direct call function 1000000 time: 9.1e-08 too fast... --------------------------------------------------------------------- signal -> function 1000000 time: 0.00556662 calls/millisecond: 179642
#include <iostream>
#include "core/signal_slot.hpp"
class Signal_receptor : public jle::signal_receptor
{
public:
Signal_receptor() = delete;
Signal_receptor(int _instance) : instance(_instance)
{
signal_resend.connect(this, &Signal_receptor::on_internal_resend);
JLE_CONNECT_METHOD(signal_resend, *this, on_internal_resend);
// connecting with this, proposed syntax ------------------------
JLE_CONNECT_THIS(signal_resend, on_internal_resend);
// ---------------------------------------------------------------
}
void on_void_method(void) {
std::cout << "on instance " << instance << " received: (void)"<< std::endl;
signal_resend.emit();
}
void on_number_method(int num) {
std::cout << "on instance " << instance << " received: " << num << std::endl;
}
void on_number_string_method(int num, const std::string& s) {
std::cout << "on instance " << instance << " received: (" << num << ", " << s << ")" << std::endl;
}
private:
int instance;
jle::signal<> signal_resend;
void on_internal_resend(void) {
std::cout << "on instance " << instance << " internal resend: (void)"<< std::endl;
}
};
void on_void_function(void)
{
std::cout << "on function " << " received: (void)" << std::endl;
}
void on_number_function(int num)
{
std::cout << "on function " << " received: " << num << std::endl;
}
void on_number_string_function(int num, const std::string& s)
{
std::cout << "on function " << " received: (" << num << ", " << s << ")" << std::endl;
}
int main(void)
{
// signal to functions
{
// no parameters
{
jle::signal<> signal;
signal.connect(on_void_function);
signal.emit();
signal.notify();
signal();
}
// one parameter
{
jle::signal<int> signal;
signal.connect(on_number_function);
signal.emit(1);
}
// two parameters
{
jle::signal<int, const std::string&> signal;
signal.connect(on_number_string_function);
signal.emit(1, "one");
signal.emit(2, "two");
}
}
// signal to method
{
{ // no parameters
jle::signal<> signal;
Signal_receptor sr1{1};
Signal_receptor sr2{2};
{
signal.connect(&sr1, &Signal_receptor::on_void_method);
JLE_CONNECT_METHOD(signal, sr2, on_void_method);
signal.emit();
}
}
{ // one parameter
jle::signal<int> signal;
Signal_receptor sr1{1};
Signal_receptor sr2{2};
{
signal.connect(&sr1, &Signal_receptor::on_number_method);
JLE_CONNECT_METHOD(signal, sr2, on_number_method);
signal.emit(1);
}
}
{ // two parameters
jle::signal<int, const std::string&> signal;
Signal_receptor sr1{1};
Signal_receptor sr2{2};
{
signal.connect(&sr1, &Signal_receptor::on_number_string_method);
JLE_CONNECT_METHOD(signal, sr2, on_number_string_method);
signal.emit(1, "one");
signal.emit(2, "two");
}
}
}
// signal to signal
{
{ // no parameters
jle::signal<> signala;
jle::signal<> signalb;
{
signala.connect(&signalb);
signalb.connect(on_void_function);
signala.emit();
}
}
{ // one parameter
jle::signal<int> signala;
jle::signal<int> signalb;
{
signala.connect(&signalb);
signalb.connect(on_number_function);
signala.emit(1);
}
}
{ // no parameters
jle::signal<int, const std::string&> signala;
jle::signal<int, const std::string&> signalb;
{
signala.connect(&signalb);
signalb.connect(on_number_string_function);
signala.emit(2, "two");
}
}
}
}
What if you destroy the signal while it is emiting? This is not a specific signal_slot pattern problem It’s possible to manage this situation in a safe way with signals, but is an incomplete solution. <+ The params could be references or raw pointers (please, do not use raw pointers) and they could be out of scope when signal is destroyed. Providing a partial solution is not a good idea. In case you do something similar, a message will be emmited on cerr It will be added asynchronous signals. They we’ll deal with this situations with no problem, but they will requiere copy on params. |
What if an exception is thrown processing a signal? At the moment, it is not managed. The exception will break the control flow not emiting the last connected signals. This looks logical. An exception is an exception. I’m thinking about the option on trapping the exception and rethrow a new one after finishing signaling. But I’m not sure it is a good idea. |
Probably you know a lambda is a function Even when you assign to a variable, the function keep existing and signal is never disconnected automatically (it’s a function). |
You can connect signals to lambdas (as connecting to functions).
If you are surprised with problems connecting to lambdas who capture scope… this is good.
Capturing scope is not a good idea and new mechanisms are provided in C++14.
3.6. Chrono
std::chrono is great, but it’s very flexible and therefore verbose.
For realtime applications, I need a precise monotonic clock.
PC clock is not precise at all. And system clock synchronized with ntp is not guaranteed to be monotonic.
jle::time_point will be monotonic ant it will synchronize with system clock slowly. Except once per day.
using jle::chono (#include "core/chrono.h"
), is a bit intrusive…
-
It will add
using namespace std::literals;
in order to use time literals as1s + 123ms
-
It will define
operator<<
onstream
forduration
To reduce verbosity, there are proposed some helpers
-
jle::chrono::duration
-
jle::chrono::now()
-
jle::chrono::make_from_date(const year&, const month&, const day&)
As usual, take a look to examples.
[TODO] micro-adjustments and one big adjustment per day
#include <iostream>
#include "core/chrono.h"
#include "core/misc.h"
int main(void)
{
{
JLE_COUT_TRACE(jle::chrono::now());
}
{
JLE_COUT_TRACE(jle::chrono::now() +1s +333ms);
}
{
JLE_COUT_TRACE(jle::chrono::now() + 10min);
}
{
JLE_COUT_TRACE(std::chrono::hours(24*360) + 5s +333ms);
}
{
auto start = jle::chrono::now() -1s -253ms;
JLE_COUT_TRACE(jle::chrono::now() - start);
}
{
JLE_COUT_TRACE(jle::chrono::make_from_date(jle::chrono::year{2015}, jle::chrono::month{4}, jle::chrono::day{13}));
}
{
JLE_COUT_TRACE(jle::chrono::today());
}
{
auto a = jle::chrono::now();
JLE_COUT_TRACE(a-a);
}
{
auto a = jle::chrono::now();
auto b = a + 1s +303ms;
JLE_COUT_TRACE(a-b);
JLE_COUT_TRACE(b-a);
JLE_COUT_TRACE(a<b);
JLE_COUT_TRACE(a>b);
JLE_COUT_TRACE(a==b);
JLE_COUT_TRACE(a!=b);
JLE_COUT_TRACE(a<=b);
JLE_COUT_TRACE(a>=b);
}
}
3.7. alarm
C++, in case of an exception, will not inform you about the call stack.
We have the exception type. It’s great, but we don’t have nested calls.
We can not force the compiler to inform about the stack call when an exception is raised, but we can give you the opportunity to stack errors information.
#include <iostream>
#include "core/alarm.h"
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << "ouch...: " << al << std::endl;
}
int main()
{
{
auto alarm = jle::alarm{JLE_HERE, "sub", JLE_SS("Description " << jle::chrono::now()), jle::al::priority::critic, jle::al::type::logic_error};
std::cout << alarm << std::endl;
}
{
try {
throw jle::alarm{JLE_HERE, "sub", JLE_SS("Description " << jle::chrono::now()), jle::al::priority::critic, jle::al::type::logic_error};
} catch(const jle::alarm& alarm) {
std::cout << alarm << std::endl;
}
}
{
try {
throw jle::alarm{JLE_HERE, "sub", JLE_SS("Description " << jle::chrono::now()), jle::al::priority::critic, jle::al::type::unknown};
} JLE_CATCH_CALLFUNCION(std::cout << , "sub_test", "description")
}
{
try {
try {
throw "this is a test";
} JLE_CATCH_RETHROW("RT", "TESTING RETHROW")
} JLE_CATCH_CALLFUNCION(std::cout << , "rt", "rethrow test")
}
return 0;
}
3.8. dbl
IEEE754 is great but isn’t infinite accurate.
There are two risk.
-
Rounds with conmutativity and asociativity operations
-
Base ten exact values that are periodic on binary
See also:
-
jle::log
-
jle::pow
-
jle::get_decimals10
Using -Wfloat-equal helps a lot, but we want to compare in a quite safe way. This is jle::dbl for.
#include <iostream>
#include "core/dbl.h"
#include "core/misc.h"
int main()
{
// this is the reason to have jle::dbl
{
std::cout << "Next is wrong..." << std::endl;
JLE_COUT_TRACE((1*(0.5-0.4-0.1) < 0.) == false)
std::cout << "Next is OK..." << std::endl;
JLE_COUT_TRACE((jle::dbl(1*(0.5-0.4-0.1)) < 0) == false)
std::cout << "We can use == ..." << std::endl;
JLE_COUT_TRACE((jle::dbl(1*(0.5-0.4-0.1)) == 0) == true)
}
// convert from double to jle::dbl
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d);
}
// operations
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d + 10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d - 10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d * 10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d / 10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d += 10.);
JLE_COUT_TRACE(d == 3.1415926535+10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d -= 10.);
JLE_COUT_TRACE(d);
JLE_COUT_TRACE(d == 3.1415926535-10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d *= 10.);
JLE_COUT_TRACE(d);
JLE_COUT_TRACE(d == 3.1415926535*10.);
}
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(d /= 10.);
JLE_COUT_TRACE(d);
JLE_COUT_TRACE(d == 3.1415926535/10.);
}
// convert from jle::dbl to double
// do you realy need it?
{
auto d = jle::dbl(3.1415926535);
JLE_COUT_TRACE(static_cast<double>(d));
}
return 0;
}
3.9. Containers
stl containers are great, but not safe.
Iterators on stl::containers could dangle and they could generate undefined behaviour.
In order to avoid it with moderate performance penalty, jle library will check the validity of iterators with "fail fast" pattern.
It looks a bad idea to keep iterators when the container has been modified, adding or removing elements. It isn’t very expensive to track iterators on these conditions.
jle will throw an exception on invalid iterator operations (instead of undefined behaviour)…
-
Iterator pointing a modifyed container (added or removed elements)
-
Iterators on empty containers
-
Derrefering end iterator
-
Comparing iterators from different containers
jle::containers are wrappers to stl containers with iterators verification
See also:
-
jle::vector
-
jle::list
-
jle::map
-
jle::set
#include "core/cont/vector.hpp"
#include "core/cont/list.hpp"
#include "core/cont/map.hpp"
#include <iostream>
#include "core/alarm.h"
int main(void)
{
{
jle::vector<int> vint;
vint.push_back(5);
vint.push_back(4);
vint.push_back(3);
vint.push_back(2);
vint.push_back(1);
std::cout << vint.size() << std::endl;
}
{
jle::map<int, std::string> mis;
mis[55] = "555555";
mis[33] = "333333";
mis[44] = "444444";
std::cout << mis.size() << std::endl;
}
{
jle::list<std::string> li;
li.push_back("00000000");
li.push_back("1111111");
li.push_back("22222222");
std::cout << li.size() << std::endl;
}
}
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << "ouch...: " << al << std::endl;
}
3.10. optional
I could contain a value, it could not contain a value…
This is a safe std::experimental::optional wrapper. No undefined behaviour
Some years ago, I wrote something similar with the name of nullable.
The right way to do it, is using algebraic data types. But, we don’t have it in C++.
I was thinking of working with algebraic data types with templates and macros, but finally, I decided to postpone it to the sdl to define data structs.
But this is a common, basic, core and very frequently needed, therefore, here it is the optional type.
#include <iostream>
#include "core/optional.hpp"
#include "core/dbl.h"
#include "core/misc.h"
namespace jle {
};
struct pr_t {
int a;
jle::dbl b;
bool operator==(const pr_t& pr) const {
return this->a == pr.a && this->b == pr.b;
}
};
int main (void)
{
{
jle::optional<pr_t> oi = pr_t{1, 3.1415926535};
oi = pr_t{1, 3.1415926535};
// checking if has a value
std::cout << oi.has_value() << std::endl;
std::cout << int(bool(oi)) << std::endl;
// accessing element
std::cout << oi->a << std::endl;
oi->a = 55;
std::cout << oi->a << std::endl;
// clear value
oi ={};
try {
std::cout << int(bool(oi)) << std::endl;
std::cout << oi->a << std::endl;
std::cout << oi->b << std::endl;
} catch(...) {
std::cout << "exception" << std::endl;
}
}
{
jle::optional<std::string> maybe_s;
maybe_s = std::string{"Hi there"};
std::cout << maybe_s.value_or("Hi here") << std::endl;
}
{
jle::optional<std::string> maybe_s;
std::cout << maybe_s.value_or("Hi here") << std::endl;
}
{
jle::optional<std::string> maybe_s;
maybe_s = std::string{"Hi there"};
std::cout << maybe_s.value() << std::endl;
}
{
jle::optional<std::string> maybe_s;
try {
std::cout << maybe_s.value() << std::endl;
} catch(...) {
std::cout << "exception" << std::endl;
}
}
{
jle::optional<std::string> maybe_s;
maybe_s = std::string{"Hi there"};
std::cout << *maybe_s << std::endl;
}
{
jle::optional<std::string> maybe_s;
try {
std::cout << *maybe_s << std::endl;
} catch(...) {
std::cout << "exception" << std::endl;
}
}
// comparison operators
{
jle::optional<pr_t> oi = pr_t{1, 3.1415926535};
jle::optional<pr_t> oi2;
oi2 = pr_t{1, 3.1415926535};
auto oi3 = jle::make_optional(pr_t{10, 1.});
JLE_COUT_TRACE(int(oi == oi2));
JLE_COUT_TRACE(int(oi != oi2));
JLE_COUT_TRACE(int(oi == oi3));
JLE_COUT_TRACE(int(oi != oi3));
}
{
auto o1 = jle::optional<std::string>{"hi"};
jle::optional<std::string> o2;
o2 = std::string{"hi"};
JLE_COUT_TRACE(int(o1 == o2));
JLE_COUT_TRACE(int(o1 < o2));
JLE_COUT_TRACE(int(o1 <= o2));
}
}
3.11. tuple
Now we have tuples on C++, but I want to write them at stream.
jle::tuple is not a wrapper, it is just an alias to std::tuple
When you include core/tuple.hpp it will inject ostream << support on tuples.
As this is a bit invasive, we created the alias jle::tuple. If you see jle::tuple, you know it has ostream<< suport
#include <iostream>
#include "core/tuple.hpp"
#include "core/alarm.h"
int main(void)
{
auto t1 = std::tuple<int, std::string>{1, "one"};
auto t2 = jle::tuple<int, std::string>{2, "two"};
if(t1 == t2)
std::cout << "yes" << std::endl;
if(t1 < t2)
std::cout << "yes" << std::endl;
JLE_COUT_TRACE(t1);
JLE_COUT_TRACE(t2);
std::cout << t1 << std::endl;
return 0;
}
namespace jle {
void alarm_msg(const jle::alarm& al)
{
std::cout << al << std::endl;
}
}
3.12. timers
It is part or message driven programming.
Only one line is necessary to configure a timer, check the example.
#include <iostream>
#include "core/alarm.h"
#include "core/signal_slot.hpp"
#include "core/timer.h"
// Function to receive timer events
void test_timer(void)
{
std::cout << jle::chrono::now() << " on_timer in fucntion" << std::endl;
}
// class to receive timer events
class Test_timer : public jle::signal_receptor {
public:
Test_timer() {
// configuring a timer for current object THIS
JLE_TIMER_THIS(7s, on_timer2)
}
void on_timer() {
std::cout << jle::chrono::now() << " on timer in class" << std::endl;
}
void on_timer2() {
std::cout << jle::chrono::now() << " on timer2 in class" << std::endl;
}
};
int main()
{
std::cout << jle::chrono::now() << " starting..." << std::endl;
// configure timer for function
JLE_TIMER_FUNCT(1s, test_timer);
// configure a timer for a method of an object
Test_timer tt;
JLE_TIMER_METHOD(2s, tt, on_timer);
// example call once
JLE_ONE_SHOOT_FUNCT(3s, [](){ std::cout << jle::chrono::now() << " CALL ONCE... " << std::endl;});
// program stop after 10s
JLE_ONE_SHOOT_FUNCT(10s, [](){ std::cout << "CLOSING... "; jle::timer::stop_main_loop();});
jle::timer::start_main_loop();
}
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << al << std::endl;
}
3.13. synchr
I don’t try to run several threads in a safe way in C++
In fact, I do the opposite.
With this library, you can run multiple threads sequentially.
If you use a third party library with other thread callbacks, or if you want to run passive waiting (and you need a thread), you can call JLE_SYNCR to avoid having more than one thread running simultaneously.
Also you have here the way to start and stop the jle mail loop.
//include::{examples}/onefile/core/ex_synchr.cpp[]
3.14. int_div0
In most hardware platforms, dividing by 0 two integers is a big problem.
On x86, it stops the program without exception or any warning.
To avoid this "undefined behavior" with bad result, you can use
JLE_HANDLE_INTDIV0 macros. See the example.
#include <iostream>
#include <stdexcept>
#include "core/int_div0.h"
#include "core/alarm.h"
int main()
{
// simple case
try
{
JLE_HANDLE_DIV0_INIT
std::cout << "one " << std::endl;
int izero = 0;
std::cout << 7 / izero << std::endl;
JLE_HANDLE_DIV0_END
}
catch(const std::exception& error)
{
std::cout << "captured in 1 " << error.what() << std::endl;
}
// two consecutives
try
{
JLE_HANDLE_DIV0_INIT
{
std::cout << "two " << std::endl;
int izero = 0;
std::cout << 7 / izero << std::endl;
}
JLE_HANDLE_DIV0_END
JLE_HANDLE_DIV0_INIT
std::cout << "three " << std::endl;
int izero = 0;
std::cout << 8 / izero;
JLE_HANDLE_DIV0_END
}
catch(const std::exception& error)
{
std::cout << "captured in 2-3 " << error.what() << std::endl;
}
// nested
try
{
JLE_HANDLE_DIV0_INIT
try
{
JLE_HANDLE_DIV0_INIT_A(A)
std::cout << "four " << std::endl;
int izero = 0;
std::cout << 7 / izero << std::endl;
JLE_HANDLE_DIV0_END_A(A)
}
catch(const std::exception& error)
{
std::cout << "captured in 4 " << error.what() << std::endl;
}
std::cout << "five " << std::endl;
std::cout << "we try AGAIN" << std::endl;
int izero = 0;
std::cout << 8 / izero;
JLE_HANDLE_DIV0_END
}
catch(const std::exception& error)
{
std::cout << "captured in 5 " << error.what() << std::endl;
}
std::cout << "End of program..." << std::endl;
return 0;
}
//---------------------------------------------------------------------------
// RECEPTOR ALARMAS SALIDA GENERAL
//---------------------------------------------------------------------------
void jle::alarm_msg(const jle::alarm& alarma)
{
std::cout << "\n\r";
std::cout << std::endl << "ALARMA SALIDA..." << alarma << std::endl ;
std::cout << alarma << std::endl ;
}
4. core / hp & AST
Full example on examples/project/idl
4.1. Aim
To enjoy!
A parser is an important tool for a programmer.
I’m also interested on methaprograming. One one to do it, and sometimes the only correct one, is creating DSLs.
In most cases, these DSLs has to be external. Working with external DSLs give you full freedom.
4.2. Tools
On tools
folder we have hpt binary. It works with hp and AST to process inputs with grammars and generates files.
On tools/qt
we have a small ui to work with grammars and templates
4.3. Main actors
4.3.1. hp
This is a simple LL(n) parser.
With it we can validate inputs with a specific grammar.
This is dynamic, therefore not fast.
It will create an AST
4.3.2. AST
As a result of parsing a file with a grammar, an AST will be generated.
We can process this AST with a c++ program, or we can use the templates transformation.
4.3.3. Templates
A template is a piece of text with functions calls.
These functions will transform the parameters, create variables and they also will return a template
4.4. Defining a grammar
4.4.1. Basic grammars
Let’s start with a common and simple exercise, parsing a math equation.
Lets start with hello world on grammars…
This grammar, accepts one or several 'a' char. Simple.
The first line, defines the initial non terminal symbol
NON Terminal symbols will be on upper case
Terminal symbols will be on lower case
As you can see, it derives by left.
Now, let’s have a sequence of 'a' followed by 'b' and a sequence of 'a' again Lets try next one, parenthesis…
It has to accept things like…
abaaa aaaabaa aba
B B ::= A b A A ::= a A A ::= a a ::= 'a' b ::= 'b'
Great.
But now, we want same quantity of 'a' at the beginning and at the end.
It has to accept things like…
aaabaaa aaaabaaa aba
But not…
aabaaa aaaaba ab
Let’s try
B B ::= a b a B ::= aa b aa B ::= aaa b aaa a ::= 'a' b ::= 'b' aa ::= 'aa' aaa ::= 'aaa'
From now on, lasts lines will indicate a valid input, rest of lines will be the grammar
This is not a full solution, and it’s not an elegant one either.
B B ::= a B a B ::= b a ::= 'a' b ::= 'b' input: aaabaaa input: aba
This looks great. Generic, concise, simple.
We can move it to parenthesis…
B B ::= ( B ) B ::= b b ::= 'b' ( ::= '(' ) ::= ')' input: (((b))) input: (b)
4.4.2. Expressions grammar
Let’s start with numbers.
EXPR EXPR ::= NUM NUM ::= d NUM NUM ::= d d ::= ([0-9]) input: 123456
Terminal symbols can be defined as regular expressions in order to simplify the rule.
To keep the example simple, we will let numbers of any size and just integers.
If terminal symbols can be written as regular expressions, then we can simplify…
EXPR EXPR ::= num num ::= ([0-9]+) input: 123456
Added one operator
EXPR EXPR ::= num operator num num ::= ([0-9]+) operator ::= ([\+|\-|\*|\/]) input: 1+2
But expressions has to accept multiple operators and numbers…
EXPR EXPR ::= num operator EXPR EXPR ::= num num ::= ([0-9]+) operator ::= ([\+|\-|\*|\/]) input: 1+2*3
And what about the parenthesis?…
EXPR EXPR ::= ( EXPR ) operator EXPR EXPR ::= ( EXPR ) EXPR ::= num operator EXPR EXPR ::= num num ::= ([0-9]+) operator ::= ([\+|\-|\*|\/]) ( ::= '(' ) ::= ')' input: (1+2)*3 input: (1*(3+2))*3+(8*9)
We could want to let spaces between elements.
EXPR EXPR ::= _ ( _ EXPR _ ) _ operator _ EXPR EXPR ::= ( _ EXPR _ ) EXPR ::= num _ operator _ EXPR EXPR ::= num num ::= ([0-9]+) operator ::= ([\+|\-|\*|\/]) ( ::= '(' ) ::= ')' _ ::= ([ |\t]*) input: ( 1+2 ) *3 input: (1* (3 +2 ) )* 3+( 8* 9 )
This grammar will produce next tree for entrance (1* (3 2 ) )* 3( 8* 9 )
Fantastic, but, what if we want to consider operator priority?…
Here it is…
EXPR EXPR ::= _ unaryoperator _ ADDS _ EXPR ::= ADDS ADDS ::= FACTS _ add_operator _ ADDS ADDS ::= FACTS FACTS ::= VAL _ mult_operator _ ADDS FACTS ::= VAL VAL ::= FUNC _ ( _ EXPR _ ) VAL ::= _ num VAL ::= VAR VAR ::= _ id FUNC ::= _ id FACTS ::= _ ( _ EXPR _ ) num ::= ([0-9]+) id ::= ([a-z|A-Z][0-9|a-z|A-Z|_]*) id ::= (_+[0-9|a-z|A-Z]+[0-9|a-z|A-Z|_]*) mult_operator ::= ([\*|\\]) add_operator ::= ([\+|\-]) unaryoperator ::= ([\+|\-]) _ ::= ([ |\t]*) ( ::= (\() ) ::= (\))
And here is the tree with correct priority for input 1+2*3
…
4.4.3. terminal especial rules
- Predefined constants
-
-
any
→ any value -
isalpha
→ letter -
islower
-
isupper
-
isdigit
-
isalnum
-
endl
-
isspace
-
isspace*
→ zero or more spaces -
isspace+
→ one or more spaces -
space_tab
→ space or tab -
All the constants can be negated with
!
-
- Regular expressions
-
-
It will be rounded by parenthesis
-
- Literals
-
-
Marked with
'
-
- Klein star
-
-
If the rule finished with *, it will be processed as a Klein star
-
4.4.4. Non terminal especial rules
- Klein star
-
-
If the rule finished with *, it will be processed as a Klein star
-
4.5. Transforming text
Validating files is quite interesting.
Parsing files and generating AST is even better.
Once you have the AST, you can do things depending of the input file.
Great!!!!
But many times, the result will be text, perhaps, another file.
Even when is not the case, generating a normalized text is an interesting option (some times a very good one) to process the input.
Generating DSLs is a good example.
That’s the main reason why I build this lib, and in these cases, the game consists on getting a text, validate it, and generating a different text.
The output could be also… a program in bytecode or c++, or embedded language.
Yes! this is external DSL
Lets see one example with the expression grammar.
Look to ##transf2→
MAIN
MAIN ::= EXPR
EXPR ::= _ VAR _ = _ EXPR ##transf2-> $(EXPR)$(__endl__)copy2:$(VAR)
EXPR ::= _ VAL _ EXPR' ##transf2-> $(VAL)$(__endl__)$(EXPR')
EXPR ::= _ unaryoperator VAL _ EXPR' ##transf2-> $(VAL)$(__endl__)$(unaryoperator)$(__endl__)$(EXPR')
EXPR ::= _ ( _ EXPR _ ) _ EXPR' ##transf2-> $(EXPR)$(__endl__)$(EXPR')
EXPR ::= _ unaryoperator ( _ EXPR _ ) _ EXPR' ##transf2-> $(EXPR)$(__endl__)$(unaryoperator)$(__endl__)$(EXPR')
EXPR' ::= POWER
EXPR' ::= FACTOR
EXPR' ::= SUM
POWER ::= _ powerop _ VAL _ POWER ##transf2-> $(VAL)$(__endl__)$(POWER)$(__endl__)$(powerop)
POWER ::= _ powerop _ VAL _ FACTOR ##transf2-> $(VAL)$(__endl__)$(powerop)$(__endl__)$(FACTOR)
POWER ::= _ powerop _ VAL _ SUM ##transf2-> $(VAL)$(__endl__)$(powerop)$(__endl__)$(SUM)
POWER ::= _ powerop _ ( _ EXPR _ ) _ EXPR' ##transf2-> $(EXPR)$(__endl__)$(mult_operator)$(__endl__)$(EXPR')
FACTOR ::= _ mult_operator _ VAL _ POWER ##transf2-> $(VAL)$(__endl__)$(POWER)$(__endl__)$(mult_operator)
FACTOR ::= _ mult_operator _ VAL _ FACTOR ##transf2-> $(VAL)$(__endl__)$(mult_operator)$(__endl__)$(FACTOR)
FACTOR ::= _ mult_operator _ VAL _ SUM ##transf2-> $(VAL)$(__endl__)$(mult_operator)$(__endl__)$(SUM)
FACTOR ::= _ mult_operator _ ( _ EXPR _ ) _ EXPR' ##transf2-> $(EXPR)$(__endl__)$(mult_operator)$(__endl__)$(EXPR')
SUM ::= _ add_operator _ VAL _ POWER ##transf2-> $(VAL)$(__endl__)$(POWER)$(__endl__)$(add_operator)
SUM ::= _ add_operator _ VAL _ FACTOR ##transf2-> $(VAL)$(__endl__)$(FACTOR)$(__endl__)$(add_operator)
SUM ::= _ add_operator _ VAL _ SUM ##transf2-> $(VAL)$(__endl__)$(add_operator)$(__endl__)$(SUM)
SUM ::= _ add_operator _ ( _ EXPR _ ) _ EXPR' ##transf2-> $(EXPR)$(__endl__)$(add_operator)$(__endl__)$(EXPR')
SUM ::= _
VAL ::= FUNC _ ( _ EXPR _ ) _ ##transf2-> $(EXPR)fun/1:$(FUNC)
VAL ::= FUNC _ ( _ EXPR _ , _ EXPR _ ) _ ##transf2-> $(EXPR)$(__endl__)$(EXPR#1)fun/2:$(FUNC)
VAL ::= num
VAL ::= VAR
VAR ::= id ##transf2-> var:$(id)
FUNC ::= id
num ::= ([0-9]*\.[0-9]+) ##transf2-> num:$(t)
num ::= ([0-9]+\.[0-9]*) ##transf2-> num:$(t)
num ::= ([0-9]+) ##transf2-> num:$(t)
id ::= ([a-z|A-Z][0-9|a-z|A-Z|_]*)
id ::= (_+[0-9|a-z|A-Z]+[0-9|a-z|A-Z|_]*)
powerop ::= (\^) ##transf2-> fun/2:$(t)
mult_operator ::= ([\*|\/]) ##transf2-> fun/2:$(t)
add_operator ::= ([\+|\-]) ##transf2-> fun/2:$(t)
unaryoperator ::= ([\+|\-]) ##transf2-> fun/1:$(t)
_ ::= ([ |\t]*) ##transf2-> $(__nothing__)
( ::= (\()
) ::= (\))
, ::= (,)
= ::= (=)
Very simple. You can generate an output for the subtree, using vars to refer the information on AST, and some predefined vars
And it will generate for input 1+2*3 +(7/9*5) +1
…
num:1
num:2
num:3
fun/2:*
num:7
num:9
fun/2:/
num:5
fun/2:*
fun/2:+
num:1
fun/2:+
fun/2:+
A small program, easy to process
You can see an example here
This is a simple calculator, with vars, functions (extensible), operator priority…
- Predefined vars
-
-
endl
-
space
-
dollar_open_par
-
close_par
-
counter
-
4.5.1. Beyond
When you need to work with complex transformation rules, you can define them outside the grammar rule.
We could call the "rules" after transfor2→
transformation templates
When we need complex transformtion templates we can write separated from grammars rules
To do that, we can insert the transformation template to a var and getting it as any other var.
[...]
EXPR ::= _ VAR _ = _ EXPR ##transf2-> $(EXPR_TPL)
[...]
__BEGIN_TEMPLATE__:: EXPR_TPL
copy2:$(VAR)
__END_TEMPLATE__::
The text between BEGIN_TEMPLATE: : <name>
and END_TEMPLATE::
is written in var <name>
A transformation template is a text with functions (or macros if you prefer) inside.
The most common case is function get
for that reason, it is special. If you don’t write a function name, it will be get
- Defined functions
$(VAR_NAME)
|
→ this is a special implicit function it’s equivalent to |
ident+
|
→ increase identation |
ident-
|
_ |
date_time
|
_ |
date
|
_ |
run
|
→ run again subtree applying current vars |
prune
|
_ |
nothing
|
_ |
set
|
_ |
copy
|
_ |
alignc
|
_ |
lmargin
|
_ |
This is a declarative language, inmutable, with rebind, and scope of closure kind.
Let see an example from idl
MAIN ::= FRAME* ##transf2->$(GENERATE_CODE) __BEGIN_TEMPLATE__:: GENERATE_CODE $(__set__ FRAME_TYPE ~ $(__set__ TYPE_OPTIONAL jle::optional<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_RECOMENDED jle::optional<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_LIST jle::list<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_WITH_DEFAULT_DATE $(BASIC_TYPE))~ ~ $(__set__ TYPE_WITH_DEFAULT $(BASIC_TYPE))~ ~ $(__set__ COMP_TYPE_NO_END $(id)::$(COMPOSED_TYPE))~ ~ $(__set__ COMP_TYPE_END $(id))~ ~ $(__run__)~ ~ $(TYPE))~ ~ $(H_FORWARD_FILE) $(H_FILE) $(CPP_FILE) __END_TEMPLATE__::
The second parameter of first function is quite long.
$(GENERATE_CODE)
will write the var content as expected (defined on BEGIN_TEMPLATE: : GENERATE_CODE…
).
When FRAME_TYPE
will be reached, it will be replaced by a lot of new set
, a run
and $(TYPE)
The ~
symbol at the beginning of the line, means… ignore spaces. And same symbol at the end, means, remove new line.
This lets us to redefine vars, and even define vars with vars inside. When AST is processed, the vars will be replaced by their value. If the value contains vars, the will be replaced by value again, and so…
Once we have declared vars, could be necessary to run again the subtree in order to apply the new defined values. This is done with run
function
Remember, functions, starts with $(
and ends with )
In some cases, inmutability will require too much computation and complex code. For example, creating a counter.
To deal with these situations (just as exceptions), next mutable functions are provided…
-
set_mut
-
get_mut
-
inc
-
dec
Lets go back to the idl example…
MAIN ::= FRAME* ##transf2->$(GENERATE_CODE) __BEGIN_TEMPLATE__:: GENERATE_CODE $(__set__ FRAME_TYPE ~ $(__set__ TYPE_OPTIONAL jle::optional<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_RECOMENDED jle::optional<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_LIST jle::list<$(BASIC_TYPE)>)~ ~ $(__set__ TYPE_WITH_DEFAULT_DATE $(BASIC_TYPE))~ ~ $(__set__ TYPE_WITH_DEFAULT $(BASIC_TYPE))~ ~ $(__set__ COMP_TYPE_NO_END $(id)::$(COMPOSED_TYPE))~ ~ $(__set__ COMP_TYPE_END $(id))~ ~ $(__run__)~ ~ $(TYPE))~ ~ $(H_FORWARD_FILE) $(H_FILE) $(CPP_FILE) __END_TEMPLATE__::
Once the AST is ready, we will run it in order to generate the output.
In this case, $(GENERATE_CODE)
will be replaced by the template content bellow.
Processing it, will declare a variable FRAME_TYPE
, will add the content of forward, h and cpp files…
MAIN ::= FRAME* ##transf2->$(GENERATE_CODE) __BEGIN_TEMPLATE__:: GENERATE_CODE $(__set__ FRAME_TYPE [...]) ~ $(H_FORWARD_FILE) $(H_FILE) $(CPP_FILE) __END_TEMPLATE__::
Let’s see the H_FILE
__BEGIN_TEMPLATE__:: H_FILE __BEGIN_FILE__::$(__file_name__).h // generated on $(__date_time__) #include <cstdint> #include <string> #include "core/tuple.hpp" #include "core/optional.hpp" #include "core/dbl.h" #include "core/cont/list.hpp" $(__set__ MODULE $(MODULE_CODE))~ $(__set__ RECORD $(RECORD_H))~ $(__set__ TUPLE $(TUPLE_H))~ $(__set__ ENUMERATION $(ENUMERATION_H))~ $(__set__ UNION $(UNION_H))~ $(__run__) $(FRAME*) $(__endl__)$(__endl__)$(__endl__) __END_TEMPLATE__::
It writes some text, declare some vars and will write $(FRAME*)
(variable defined in AST)
One example with counters…
__BEGIN_TEMPLATE__:: COUNT_TUPLE_FIELDS
$(__nothing__ count number of fields)~
$(__set_mut__ PCOUNTER 0)~
$(__set__ F_NO_NAMED_NOEND $(__inc__ PCOUNTER)$(F_NO_NAMED))~
$(__set__ F_NO_NAMED_END $(__nothing__))~
$(__run__)~
__END_TEMPLATE__::
[...]
$(COUNT_TUPLE_FIELDS)~
$(__nothing__ write fields with counter)~
$(__set__ F_NO_NAMED_NOEND $(FULL_TYPE) p$(__get_mut__ PCOUNTER)$(__dec__ PCOUNTER),$(__endl__)$(F_NO_NAMED))~
$(__set__ F_NO_NAMED_END $(FULL_TYPE) p$(__get_mut__ PCOUNTER)$(__dec__ PCOUNTER))~
$(__run__)
You can see a full example here idl
4.6. Tooling
Great, we have a LL(n) parser, with rules and templates to generate an output.
4.6.1. Grammar and templates processor
This has been used to create hpt
This is a small program who lets us to define the input files, and grammar with template files.
It will run all, and will produce the result.
4.6.2. Grammar and template editor
This is a simple Qt program who helps us to write grammars, templates and debug it
You can work with several grammars and choose the proper one.
You can edit the grammar and templates
You can edit the input
You can edit show the tree with transformations applied
You can show the output
5. net
5.1. http_server
Http server for embedding.
It can be used for REST
At the moment, it is a wrapper from fossa https://github.com/cesanta/fossa
// Http message format struct http_message { struct ns_str message; // Whole message: request line + headers + body
struct ns_str proto; // "HTTP/1.1" -- for both request and response // HTTP Request line (or HTTP response line) struct ns_str method; // "GET" struct ns_str uri; // "/my_file.html" // For responses, code and response status message are set int resp_code; struct ns_str resp_status_msg;
// * // * Query-string part of the URI. For example, for HTTP request // * GET /foo/bar?param1=val1¶m2=val2 // * | uri | query_string | // * // * Note that question mark character doesn't belong neither to the uri, // * nor to the query_string // * struct ns_str query_string;
// Headers struct ns_str header_names[NS_MAX_HTTP_HEADERS]; struct ns_str header_values[NS_MAX_HTTP_HEADERS];
// Message body struct ns_str body; // Zero-length for requests with no body
#include "net/http_server.h"
#include "core/timer.h"
void jle::alarm_msg(const jle::alarm& al)
{
std::cout << al << std::endl;
}
void on_request_received(const jle::shared_ptr<jle::net::http::Request>& rq)
{
static int i = 0;
std::cout << "received request on port " << rq->listen_port << std::endl;
std::cout << rq->request_msg << std::endl;
rq->send_response(jle::net::http::response_ok(JLE_SS("All is OK " << ++i << " on port " << rq->listen_port)));
}
int main(int /*argc*/, char ** /*argv[]*/)
{
auto start_server = [] (const std::string& port) {
std::cout << "starting server on port " << port << std::endl;
auto server = std::make_shared<jle::net::http::Server>(port);
server->signal_request_received.connect(on_request_received);
return server;
};
// run two servers
auto server1 = start_server("8000");
auto server2 = start_server("8001");
// stop the server after 10s
JLE_ONE_SHOOT_FUNCT(10s, []() {
std::cout << "STOPPING THE PROGRAM..." << std::endl;
jle::timer::stop_main_loop();
});
// start jle main loop
jle::timer::start_main_loop();
return 0;
}
6. Tools
6.1. hpt
Humble Parser Template
This is a small console program working with jle/hp
It will receive the grammar files and input file
usage...
hpt <gram-file> <2gram-file> <input-file>
It will run hp in order to build the AST and it will run the AST making transformations as defined in grammar and template files.
Input example…
/* multiline
comment
*/
module idl.pr {
t_simple {
i : int32
j : string
s : string
/* multiline
comment
*/
}
t_inline { i: int32, s: string }
module nested { t_inline { i: int32, s: string } }
t_inline2 {
i: int32, i64: int64
s: string
}
t_tuple { int32, string }
t_tuple2 {
int32
string
}
t_color // enumeration
{
| red
| blue
| green
| orange
| pink
| white
| black
}
// type modifiers
t_type_modifiers
{
// commentary
int_field : int32 // mandatory
string_field : string
opt_string : string? // optional
rec_float : double! // recommended
def_string0 : string[<"hi">] // default value
def_string1 : string[<"there">][<2015-10-23>] // new mandatory since...
string_list : string[]
int_list : int32[]
}
t_union
{
| s : string
| i : int32
| d : double
}
} // module idl.pr {
t_composed {
i2 : int32
comp : idl.pr.t_simple
}
It will produce forwarded header, header and cpp code.
A full example…
6.2. Grammar and template editor
Part of jle_cpp_tk/parser
This is a simple Qt program who helps us to write grammars, templates and debug it
You can work with several grammars and choose the proper one.
You can edit the grammar and templates
You can edit the input
You can edit show the tree with transformations applied
You can show the output
7. Examples
7.1. Calculator
This is a small example using these tools.
You can run assignments
y=x=-1+2*(j=3)
And simple operations
y*2.5
7*9+(7-3*4+(5*x+1)-x/2)/y
// --------------------------------------------
{
// first compilation step
std::string asm_code;
{
jle::hp::AST_node_item ast_root("");
bool result=false;
std::string resultTest;
std::tie(result, resultTest, ast_root) =
h_parser.multi_parse("y=x=-1+2*(j=3)");
if (result == false)
{
std::cout << std::endl << "SYNTAX ERROR " << resultTest << std::endl;
return 0;
}
asm_code = ast_root.value;
}
// finishing compilation
asembler.compile(asm_code);
// evaluation
auto result = amachine.eval();
std::cout << std::endl << "RESULT... " << result << std::endl;
std::cout << std::endl << std::endl << "x == " << amachine.get_value_from_heap("x");
std::cout << std::endl << "y == " << amachine.get_value_from_heap("y");
std::cout << std::endl << "z == " << amachine.get_value_from_heap("z");
std::cout << std::endl << "j == " << amachine.get_value_from_heap("j") << std::endl;
}
7.2. idl
we will generate the cpp source code from an idl (external DSL to define structs)
This is an example of input
/* multiline
comment
*/
module idl.pr {
t_simple {
i : int32
j : string
s : string
/* multiline
comment
*/
}
t_inline { i: int32, s: string }
module nested { t_inline { i: int32, s: string } }
t_inline2 {
i: int32, i64: int64
s: string
}
t_tuple { int32, string }
t_tuple2 {
int32
string
}
t_color // enumeration
{
| red
| blue
| green
| orange
| pink
| white
| black
}
// type modifiers
t_type_modifiers
{
// commentary
int_field : int32 // mandatory
string_field : string
opt_string : string? // optional
rec_float : double! // recommended
def_string0 : string[<"hi">] // default value
def_string1 : string[<"there">][<2015-10-23>] // new mandatory since...
string_list : string[]
int_list : int32[]
}
t_union
{
| s : string
| i : int32
| d : double
}
} // module idl.pr {
t_composed {
i2 : int32
comp : idl.pr.t_simple
}
We have line and multiline comments, we have simple and inlined struct definitions, nested modules, tuples, enumerations, unions, composition, type modifiers…
Union type will be mapped in cpp combining enum with struct. Then you can use it simulating unions on ADT combined with cpp switch |