24 December 2018

After Now Archeology

Although it has passed 4 years from his original thesis about puzzle game design. Jakko has defended a complete document and began a game.

The blog a thesis can be found at :  http://afternowarcheology.blogspot.fi/

The research refers to several important books/games in the field.

21 December 2018

OpenGrok

Open source project from Oracle.

It retrieves information from different control version source code and is indexed with Apache Lucene, making a fast solution for searching (and keep updating).



https://github.com/oracle/opengrok/wiki/How-to-setup-OpenGrok
http://demo.opengrok.org/
https://github.com/oracle/opengrok


18 December 2018

Most interesting Socket Options

  • TCP_NODELAY
  • SO_SNDBUF
  • SO_RCVBUF
  • Don't forget timeout !

      socket = new Socket(host, port);
      sock.setSoTimeout(5 * 1000); //milliseconds

      socket.setTcpNoDelay(true);
      socket.setSendBufferSize(1000);
      socket.setReceiveBufferSize(1000);


https://eklitzke.org/the-caveats-of-tcp-nodelay
http://normanmaurer.me/presentations/2013-jax-networking-on-jvm/#/41

Don't want to Hassle

    There are a few frameworks to rescue....

12 December 2018

Bfile++

Quick analysis as an alternative and abstract layer for Oracle BFILE.

- http://www.jcraft.com/jsch/
  JSch library
  https://docs.spring.io/spring-integration/reference/html/sftp.html
  Under the covers, the SFTP Session Factory relies on the JSch library to provide the SFTP capabilities.

- https://commons.apache.org/proper/commons-net/
  FTPFile (Apache Commons Net 3.6 API)

- Commons VFS
  https://commons.apache.org/proper/commons-vfs

- Xodus (JetBrains)
  https://github.com/JetBrains/xodus
  It has three types of database: a key-value storage, an entity-store and a database for files and streams.

- Flexible Storage System (FS2)
  https://github.com/robertjchristian/fs2
  (un solo developer :( esta buena la idea )

Servers:
(out of scope)

- http://www.alluxio.org/ (service)
- https://www.syncany.org/ (tool)


More info:

https://docs.oracle.com/cd/B28359_01/java.111/b31224/oralob.htm#i1059941
https://stackoverflow.com/questions/51503297/managing-oracle-bfiles
http://www.orafaq.com/wiki/BFILE
https://docs.oracle.com/cd/B28359_01/java.111/b31224/oralob.htm#i1059941

11 December 2018

Artist Entities for Coding Books

It is not very common to see in programming books writers, musicians, artist as examples in code samples (usually as entities)

Some examples:

- Get Programming with Haskell

 -  Lovecraft

names = [("Ian", "Curtis"), 
         ("Bernard","Sumner"), 
         ("Peter", "Hook"), 
         ("Stephen","Morris")]


- If Hemingway Wrote JavaScript

    - Ernest Hemingway
    - William Shakespeare
    - André Breton
    - Roberto Bolaño
    - Dan Brown

Table of contents:

Fibonacci
1 Ernest Hemingway
2 William Shakespeare
3 André Breton
4 Roberto Bolaño
5 Dan Brown
Factorial
6 Jack Kerouac
7 Jane Austen
8 Samuel Johnson
9 Sir Arthur Conan Doyle
10 James Joyce
Happy Numbers
11 J.D. Salinger
12 Tupac Shakur
13 Virginia Woolf
14 Geoffrey Chaucer
15 Vladimir Nabokov
Prime Numbers
16 Jorge Luis Borges
17 Lewis Carroll
18 Douglas Adams
19 Charles Dickens
20 David Foster Wallace
Say It
21 Sylvia Plath
22 Italo Calvino
23 J.K. Rowling
24 Arundhati Roy
25 Franz Kafka
Poetic Interludes
1 Edgar Allan Poe
2 William Shakespeare
3 Dylan Thomas
4 Walt Whitman

10 December 2018

Look into JMS filestore

For many reasons sometimes is necessary to lookup JMS filestore directly, this can be achieved through:

1.
java weblogic.store.Admin

2.
storeadmin-> openfile -store FileStore-FileStore1 -dir /tmp/filestore-local

3.
storeadmin-> dump -store FileStore-FileStore1 -out /Users/German/tmp/filestore-local-dump.xml -conn -deep
4.
list -dir /tmp/filestore-local

It can be used for compressing the store as well.

more info:
https://docs.oracle.com/middleware/1221/wls/STORE/admin.htm#STORE363

06 December 2018

Ludic HTTP

418 I'm a teapot

The HTTP 418 I'm a teapot client error response code indicates that the server refuses to brew coffee because it is a teapot. This error is a reference of Hyper Text Coffee Pot Control Protocol which was an April Fools' joke in 1998

451 Unavailable For Legal Reasons:

I just discovered that HTTP status code 451 "Unavailable For Legal Reasons" exists. This number, in the 4xx "Client errors" range, is a reference to Ray Bradbury's Fahrenheit 451 in which books are illegal...

04 December 2018

WLS Shortcuts:

Useful WebLogic Server shortcuts of Admin console:


Servers -> YOUR_SERVER -> General -> Tuning -> Advanced -> Self Tuning Thread Maximum Pool Size: 400 (default)
 

Servers -> YOUR_SERVER -> Monitoring -> General : jdk, os, etc.
 

Servers -> YOUR_SERVER -> Protocols -> Channels : t3, t3s, http, https, etc.

03 December 2018

Optimizar o No Optimizar

desde el 2008.... y fines de 2018 se mantiene...

Viendo un post en el blog http://gpicon.blogspot.com/2007/11/optimizando-solucion-problema-de.html mencionan sobre optimizar y ver comentarios tan radicales como "si el problema es ficticio es una tontera optimizar", se ve varios puntos de vistas que podemos ver en la vida real.

1. si el problema no fuese ficticio entonces no trabajemos en él, entonces la mayoria de las matematicas no existirian hasta encontrarle alguna aplicacion entonces las matematicas no serian lo que son si ese fuese el paradigma (por suerte no hay tan cortos de mente en la historia de la civilización humana).

2. como sale en el triángulo de (... ver cuaderno), segun el tiempo que se le invierte podemos optimizar algun algoritmo (tarea, metodo, etc.), segun la dificultad y los tiempos (como tambien mencionan en otro post, que la Gantt siga bajo fechas estimadas),

3. hoy en dia hay varios paradigmas de metodos agiles (cual utilizo cuando desarrollo) que si el algoritmo funciona dentro de tiempos y uso de recursos razonables entonces dejemoslo ahi y vamos por otro requerimiento, despues podemos volver a optimizar y mejorar (siempre se puede), pero tenemos que terminar otras cosas antes (bueno el proyecto en si).

4. Esto me recuerda a la trilogia de articulos clasicos de Knuth (casi imposible mencionarlo), sobre la curva/transaccion de teoria versus practica, donde es dificil definir el limite hasta donde va cual paradigma y donde ambas hacen simbiosis.

Volviendo al blog inicial, creo que esos comentarios de optimizar es malo estan totalmente errados y deberian intentar meditar mejor sus dichos para poder mejorar el area de desarrollo (en todo ambito).

Creo que acá puede ser mal entendido el dicho de Knuth:

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." (Knuth, Donald. Structured Programming with go to Statements, ACM Journal Computing Surveys, Vol 6, No. 4, Dec. 1974. p.268.).

La optimización debe ser analizada si realmente es necesaria gastar esfuerzo en algo que impacto en el rendimiento de un algoritmo ya que los cambios marginales se pueden dejar ya que sólo aumento el nivel de riesgo que se dañe lo ya logrado.

27 November 2018

Object Pooling

Object Pooling, a common question, but with not easy to find solutions :

https://stackoverflow.com/questions/43735067/manage-client-socket-pool
https://stackoverflow.com/questions/939734/tips-for-using-commons-pool-in-production
https://github.com/roma/roma-java-client/blob/master/java/client/src/main/java/jp/co/rakuten/rit/roma/client/SocketPool.java

https://commons.apache.org/proper/commons-pool/
http://www.vibur.org/vibur-object-pool/
http://danielw.cn/fast-object-pool/


CEStreamExhausted anti-pattern

This exception is thrown when EOF is found. Literally is an anti-pattern (like a goto)

https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/misc/CEStreamExhausted.java

/** This exception is thrown when EOF is reached */
public class CEStreamExhausted extends IOException { };


Which is executed at line 117 de BASE64Decoder :

https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/misc/BASE64Decoder.java

do {
   i = inStream.read();
   if (i == -1) {
      throw new CEStreamExhausted();
   }
} while (i == '\n' || i == '\r');


Therefore it's a false-positive if any APM raise it.

--------------------------------------------------------------------------------

Me faltó agregar que la clase que llama a la lanza la excepcion en cuestion es: CharacterDecoder.java

Y lo que hace es tener un loop infinito while(true) que se sale de ahi al atrapar dicha exception.

Por lo tanto queda demostrado que es un falso-positivo.

while (true) {

int length;


try {

length = decodeLinePrefix(ps, bStream);

for (i = 0; (i+bytesPerAtom()) < length; i += bytesPerAtom()) {

decodeAtom(ps, bStream, bytesPerAtom());

totalBytes += bytesPerAtom();

}

if ((i + bytesPerAtom()) == length) {

decodeAtom(ps, bStream, bytesPerAtom());

totalBytes += bytesPerAtom();

} else {

decodeAtom(ps, bStream, length - i);

totalBytes += (length - i);

}

decodeLineSuffix(ps, bStream);

} catch (CEStreamExhausted e) {

break;

}


26 November 2018

Log4j config ?

A way to check if log4j is configurated at runtime.

log4j IsConfigured ::: 

    /**
     * Returns true if it appears that log4j have been previously configured. This
     * code checks to see if there are any appenders defined for log4j which is the
     * definitive way to tell if log4j is already initialized
     */
    private static boolean isConfigured() {
        Enumeration appenders = Logger.getRoot().getAllAppenders();
        if (appenders.hasMoreElements()) {
            return true;
        } else {
            Enumeration loggers = LogManager.getCurrentLoggers();
            while (loggers.hasMoreElements()) {
                Logger c = (Logger) loggers.nextElement();
                if (c.getAllAppenders().hasMoreElements())
                    return true;
            }
        }
        return false;
    }



http://wiki.apache.org/logging-log4j/UsefulCode
http://logging.apache.org/log4j/1.2/xref-test/org/apache/log4j/defaultInit/TestCase4.html
https://web.archive.org/web/20130401061324/http://dsiutils.dsi.unimi.it/
https://web.archive.org/web/20081211154329/http://dsiutils.dsi.unimi.it/docs/it/unimi/dsi/Util.html
https://web.archive.org/web/20110303190435/http://www.screaming-penguin.com/node/7622

16 November 2018

Loopback

Loopback in WLS

<Oct 18, 2018 6:14:01 PM EDT> <Notice> <WebLogicServer> <BEA-000355> <Thread "ListenThread.Loopback" listening on port 7001, ip address 127.0.0.1>


14 November 2018

Oxímoron

El oxímoron (del griego ὀξύμωρον, oxymoron, en latín contradictio in terminis), dentro de las figuras literarias en retórica, es una figura lógica que consiste en usar dos conceptos de significado opuesto en una sola expresión,1
Ejemplo: «Festina lente», ‘apresúrate lentamente’ (César Augusto, 63 a. C.-14 d. C.)


https://es.wikipedia.org/wiki/Ox%C3%ADmoron

1. https://www.retoricas.com/2009/06/4-ejemplos-de-oximoron.html

08 November 2018

diff between SoftReference and WeakReference

A complete answer can be found at: https://stackoverflow.com/questions/299659/whats-the-difference-between-softreference-and-weakreference-in-java



From Understanding Weak References, by Ethan Nicholas:
Weak references
A weak reference, simply put, is a reference that isn't strong enough to force an object to remain in memory. Weak references allow you to leverage the garbage collector's ability to determine reachability for you, so you don't have to do it yourself. You create a weak reference like this:
WeakReference weakWidget = new WeakReference(widget);
and then elsewhere in the code you can use weakWidget.get() to get the actual Widget object. Of course the weak reference isn't strong enough to prevent garbage collection, so you may find (if there are no strong references to the widget) that weakWidget.get() suddenly starts returning null.
...
Soft references
A soft reference is exactly like a weak reference, except that it is less eager to throw away the object to which it refers. An object which is only weakly reachable (the strongest references to it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is softly reachable will generally stick around for a while.
SoftReferences aren't required to behave any differently than WeakReferences, but in practice softly reachable objects are generally retained as long as memory is in plentiful supply. This makes them an excellent foundation for a cache, such as the image cache described above, since you can let the garbage collector worry about both how reachable the objects are (a strongly reachable object will never be removed from the cache) and how badly it needs the memory they are consuming.
And Peter Kessler added in a comment:
The Sun JRE does treat SoftReferences differently from WeakReferences. We attempt to hold on to object referenced by a SoftReference if there isn't pressure on the available memory. One detail: the policy for the "-client" and "-server" JRE's are different: the -client JRE tries to keep your footprint small by preferring to clear SoftReferences rather than expand the heap, whereas the -server JRE tries to keep your performance high by preferring to expand the heap (if possible) rather than clear SoftReferences. One size does not fit all.

07 November 2018

Sonarqube cheat sheet

SonarQube Best open source static analyzer.
(IMHO :-) ).

1.
How to create custom plugins for Sonar

https://devwebcl.blogspot.com/2018/10/custom-sonarqube-plugins-63.html

2.
default port: 9000
http://127.0.0.1:9000/

http://localhost:9000/

3.
Maven command line:

mvn sonar:sonar -Dsonar.host.url=http://127.0.0.1:9000 -Dsonar.login=d676f79d8ba83cdcf69f38f8471f0284ee242e09

4.
Quality Gates

Quality Gates are the best way to ensure that standards are met and regulated across all the projects in your organization. Quality Gates can be defined as a set of threshold measures set on your project like Code Coverage, Technical Debt Measure, Number of Blocker/Critical issues, Security Rating, etc.

5.
Quality Profiles

A quality profile in Sonar consists of: A set of activated coding rules among +600 available (PMD, Checkstyle and FindBugs): an activation level (mandatory or optional) and parametrization for each rule.

6.
wsdl jar
to be tested jar wsdl client proxies :

           
                      <plugin>
                <groupId>org.jvnet.jax-ws-commons</groupId>
                <artifactId>jaxws-maven-plugin</artifactId>
                <version>2.3</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>wsimport</goal>
                        </goals>
                        <configuration>
                            <wsdlDirectory>${basedir}/src/main/resources/wsdl/</wsdlDirectory>
                            <wsdlLocation>wsdl/endpoint_sample_wsdl.wsdl</wsdlLocation>

                            <sourceDestDir>${basedir}/src/main/java</sourceDestDir>

7. sonnar-scanner cli

sonar-scanner -Dsonar.projectKey=my-best-project


8. sonar-project.properties
to active a project and be used by sonar cli.
Create an user and generate token, then add to properties file:

# must be unique in a given SonarQube instance
sonar.projectKey=my:project
sonar.login=f86a58b7f520bfc6bbdb5bf03a0671ce64860f0c
sonar.java.binaries=target
 

#to avoid java parsing
sonar.exclusions=**/*.java


9. important not default rules

- https://rules.sonarsource.com/java/RSPEC-3749
  Members of Spring components should be injected

- https://rules.sonarsource.com/java/RSPEC-4288
  Spring components should use constructor injection

 

31 October 2018

Custom sonarqube-plugins 6.3

A simple example I made for custom plugin in Java: https://github.com/devwebcl/sonarqube-plugins
The visitor Pattern is the core of the solution, similar to Java Parser.


sonarqube-plugins 6.3

This example demonstrates how to write Custom Rules for the SonarQube Java Analyzer (aka SonarJava).
It requires to install SonarJava 4.7.1.9272 on your SonarQube 5.6+
--> actually, it needs a newer version, please check pom.xml
Class FilenamepathDescription
MyFirstCustomCheck.java/src/test/files/A test file, which contains Java code used as input data for testing the rule
org.sonar.template.java.checks. MyFirstCustomCheckTest.java/src/test/javaA test class, which contains the rule's unit test
org.sonar.template.java.checks. MyFirstCustomCheck.java/src/main/javaA rule class, which contains the implementation of the rule.

import org.sonar.api.Plugin;

/**
 * Entry point of your plugin containing your custom rules
 */
public class MyJavaRulesPlugin implements Plugin {
This class is the entry point for the SONAR plugin. This class is extended from org.sonar.api.Plugin class. This class includes server extension which gets instanciated during sonarqube startup, and batch extensions which get instantiated during the code analysis.

/**
 * Declare rule metadata in server repository of rules.
 * That allows to list the rules in the page "Rules".
 */
 public class MyJavaRulesDefinition implements RulesDefinition {
This class is a Server extension that gets instanciated at the time of sonarqube startup. The repository name and supported language name is mentioned in this class
    // server extensions -> objects are instantiated during server startup
    context.addExtension(MyJavaRulesDefinition.class);

    // batch extensions -> objects are instantiated during code analysis
    context.addExtension(MyJavaFileCheckRegistrar.class);

/**
 * Provide the "checks" (implementations of rules) classes that are going be executed during
 * source code analysis.
 *
 * This class is a batch extension by implementing the {@link org.sonar.plugins.java.api.CheckRegistrar} interface.
 */
@SonarLintSide
public class MyJavaFileCheckRegistrar implements CheckRegistrar {
This class is the batch extension which gets instanciated during the code analysis. This class registers all custom rule classes.

/*Rule Activation

The second things to to is to activate the rule within the plugin. To do so, open class RulesList (org.sonar.samples.java.RulesList). In this class, you will notice methods GetJavaChecks() and GetJavaTestChecks(). These methods are used to register our rules with alongside the rule of the Java plugin.*/
public final class RulesList {
This class lists all custom rules and provides the list to the CustomJavaFileCheckRegistrar class to register them with sonarqube

17 October 2018

Eclipse MAT++

1. Retained vs Shallow heap

Shallow is the real size of the object in the heap, meanwhile Retained is the complete memory space used by all the references.



https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Fconcepts%2Fshallowretainedheap.html

https://plumbr.io/blog/memory-leaks/how-much-memory-what-is-retained-heap


2. Heap diagram:

Many times, it is not very known the states of objects in Heap memory:

https://mechanical-sympathy.blogspot.com/2013/07/java-garbage-collection-distilled.html


3.  “GC root path”.
Mat can calculate the full path from GC to any class, this way we can see big size objects.

Now, MAT will start calculating the memory graph to show the paths to the GC root where this instance is referenced. This will show up with another page, showing the references as below:




https://dzone.com/articles/java-out-of-memory-heap-analysis


4. MAT resources
A small good tip is about the necessary resources to run mat.

As a rough estimate if the number of objects is N and the number of classes C, it might take at least T bytes to parse and build the dominator tree where:

T ≈ N * 28.25 + C * 1000 + P
P is the space used by the DTFJ or HPROF parsers. For a PHD file, the space could be:
P ≈ C * 1000
Memory Analyzer uses additional memory for caching index files, so performance will be better if there is more memory available than the minimum required to parse a dump.

from: https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Ftasks%2Fconfigure_mat.html

5. command line:
Sometimes a good choice to process several heapdumps is to execute thru command line:

sh ./ParseHeapDump.sh /heapdumps/java_7935_heapdump_4p.hprof org.eclipse.mat.api:suspects org.eclipse.mat.api:overview org.eclipse.mat.api:top_components 

 It will create the following reports:

- org.eclipse.mat.api:suspects
- org.eclipse.mat.api:overview
- org.eclipse.mat.api:top_components

02 October 2018

JMX Remote (visualvm)

Arguments for JVM thus VisualVM can connect remotely JMX :

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false


30 September 2018

TCP/IP Illustrated

RFC793 has several diagrams, in particular, from book TCP/IP Illustrated, Vol. 1: The Protocols, and concise information for this protocol.

IBM has a good replication:



Table 1. TCP state transition description table
TCP connection state Abbreviation in MVS™ console Abbreviation in TSO or UNIX shell Description
LISTEN Listen Listen Waiting for a connection request from a remote TCP application. This is the state in which you can find the listening socket of a local TCP server.
SYN-SENT SynSent SynSent Waiting for an acknowledgment from the remote endpoint after having sent a connection request. Results after step 1 of the three-way TCP handshake.
SYN-RECEIVED SynRcvd SynRcvd This endpoint has received a connection request and sent an acknowledgment. This endpoint is waiting for final acknowledgment that the other endpoint did receive this endpoint's acknowledgment of the original connection request. Results after step 2 of the three-way TCP handshake.
ESTABLISHED Estblsh Establsh Represents a fully established connection; this is the normal state for the data transfer phase of the connection.
FIN-WAIT-1 FinWt1 FinWait1 Waiting for an acknowledgment of the connection termination request or for a simultaneous connection termination request from the remote TCP. This state is normally of short duration.
FIN-WAIT-2 FinWt2 FinWait2 Waiting for a connection termination request from the remote TCP after this endpoint has sent its connection termination request. This state is normally of short duration, but if the remote socket endpoint does not close its socket shortly after it has received information that this socket endpoint closed the connection, then it might last for some time. Excessive FIN-WAIT-2 states can indicate an error in the coding of the remote application.
CLOSE-WAIT ClosWt ClosWait This endpoint has received a close request from the remote endpoint and this TCP is now waiting for a connection termination request from the local application.
CLOSING Closing Closing Waiting for a connection termination request acknowledgment from the remote TCP. This state is entered when this endpoint receives a close request from the local application, sends a termination request to the remote endpoint, and receives a termination request before it receives the acknowledgment from the remote endpoint.
LAST-ACK LastAck LastAck Waiting for an acknowledgment of the connection termination request previously sent to the remote TCP. This state is entered when this endpoint received a termination request before it sent its termination request.
TIME-WAIT TimeWt TimeWait Waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request.
CLOSED Closed Closed Represents no connection state at all.


Also, a sequence diagram, where Wikipedia is a good contribution (as usual):

- open: 3 way
- close: 4 way




http://www.icir.org/christian/sock.html

https://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml

http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm


Blog Archive

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.