I understand that Nucoder encodes the php script into php byte codes.

What is exactly the encoding/encryption algorithm used in NuCoder?

I need to make sure that the encoding/encryption algorithm used does not violate the regulation set by some regions.

In terms we understand it, encoding does not mean encryption. If you compile for example C program with regular C compiler, you'll get CPU opcodes suitable for the target platform CPU. It's known as binary file that operating system can recognize, load and execute.
With nucoder you get quite similar binary file that is suitable for running under PHP virtual machine which supports its own opcodes. The file format is recognzed by PhpExpress (the loader) and it can load the file and let php vm execute the opcodes.

There is no encryption until you protect your encoded files with license. In latter case nucoder uses AES-128bit. It's the same cypher algorithm that is used by your browser when you work with HTTPS web sites.

As far a I heard some contries (for example China) do not allow any encryption with keys longer than 56bit. It's a problem then because there is no such thing as AES 56bit, only DES and it's too weak.