Estándar

Monitoring Apache webserver and MySql

Groovy scrips

http://www.mkyong.com/java/how-to-run-a-task-periodically-in-java/

http://mrhaki.blogspot.com.es/2009/11/groovy-goodness-run-code-at-specified.html

https://gist.github.com/kdabir/3176945

http://groovy-almanac.org/example-timer-and-timertask-in-groovy/

http://crazy4groovy.blogspot.com.es/2013/01/simple-timer-timertask.html

 

Data storage

http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb

  • https://github.com/Glavin001/Cobra
  • https://www.quora.com/What-is-the-best-way-to-store-time-series-data-in-MongoDB

https://www.mongodb.com/customers/server-density?_ga=1.111702214.317250607.1443618784

http://www.hindawi.com/journals/ijdsn/2013/867693/

Web Analytics using MongoDB Aggregation Framework

MongoDB

http://snmaynard.com/2012/10/17/things-i-wish-i-knew-about-mongodb-a-year-ago/

https://blog.codecentric.de/en/2014/01/realtime-analytics-mongodb-nodejs-smoothiecharts/

http://s3.amazonaws.com/info-mongodb-com/10gen-MongoDB_Operations_Best_Practices.pdf

Cube

http://square.github.io/cube/

Groovy

https://blog.sagaoftherealms.net/?p=176

http://blog.mongodb.org/post/18510469058/grails-in-the-land-of-mongodb

Node.js

http://bigspaceship.github.io/blog/2014/05/14/how-to-create-a-rest-api-with-node-dot-js/

http://www.robinwieruch.de/node-js-scheduling-periodic-tasks-with-agenda/

https://github.com/mikejihbe/metrics

http://stackoverflow.com/questions/9565912/convert-the-output-of-os-cpus-in-node-js-to-percentage

https://www.npmjs.com/package/nsutil

Python:

https://github.com/hcooper/check_mk-plugins

https://github.com/monitisexchange

Apache

http://httpd.apache.org/docs/2.4/mod/mod_status.html

 

local.conf

<VirtualHost *:80>
	DocumentRoot "/var/www/html"
	ServerName local.tld
	ServerAlias *.local.tld
	<Directory "/var/www/html">
		Options All
		AllowOverride All
		#Order allow,deny
		#Allow from all
		# Apache 2.4
		Require all granted
	</Directory>
	<Location /server-status>
		SetHandler server-status
		Require ip 127.0.0.1
		Require ip ::1
		Require ip 192.168.0.114
		Require ip 192.168.0.135
	</Location>
</VirtualHost>

 

https://github.com/fr3nd/apache-top

https://exchange.nagios.org/directory/Plugins/Web-Servers/Apache/check_apache2-2Epy/details

https://github.com/hcooper/check_mk-plugins/blob/master/mod_status/mod_status-dump.py

http://www.wellho.net/mouth/3019_Apache-httpd-Server-Status-monitoring-your-server.html

https://github.com/dioubernardo/apacheServerStatusCharts

http://pastebin.com/9iGkjSTv

 

How it works

There is an Apache module (mod_status) responsible for presenting the current view of Apache key parameters and a snapshot of all its request handlers with their statuses. The following lines in Apache’s configuration file present an example of mod_status switched on:

<Location /server-status>
	SetHandler server-status
	Order deny,allow
	Deny from all
	Allow from 127.0.0.1
</Location>

The statistics are provided in the form of a webpage – you can have a look at such a page of the Apache Project itself here:http://www.apache.org/server-status. Anturis Apache Monitor is able to read and understand this web page (for example: http://127.0.0.1/server-status), and present it on the Anturis Console.

Currently Anturis Apache monitor collects the following metrics:

  • aggregate CPU and memory consumption of all Apache processes
  • number of busy workers
  • number of idle workers
  • average number of requests per second*
  • average number of Kbytes served per second*
  • average number of Kbytes per request*
  • CPU usage by all Apache workers*

*provided ExtendedStatus is switched on in Apache configuration.

 

Monitoring web traffic

A good starting point for troubleshooting website problems is to check whether traffic was or is unusual. An unexpected surge in traffic may be caused by many reasons varying from a DOS attack to a marketing campaign that a sys administrator was unaware of. Anturis Apache Monitor (when mod_status and ExtendedStatus is enabled in your Apache server configuration) provides the following information about the overall traffic:

  • number of requests per second
  • number of bytes served per second
  • average number of bytes per request

Comparing these values with historical data may reveal any abnormalities, such as unusually high or low traffic being processed.

ver: http://piwik.org/

Monitoring Apache processes and threads

Latest Apache versions implement hybrid multi-threaded and multi-process models to serve requests. It means that you will see multiple Apache processes running. Each process may contain multiple threads. The number of processes and the number of threads per process, as well as a forking policy and other parameters, are governed by Apache configuration directives. Each request processing thread is called aworker.

Apache self-regulates the number of workers and maintains a pool of idle workers so as to have some ready to process traffic at any time. Anturis Apache Monitor allows you to keep an eye on the number of busy and idle workers.

Watching the number of busy and idle workers is a proactive way to catch Apache configuration problems early enough. Anturis Apache Monitoring tool will let you discover, for example, these problems:

  • insufficient workers for the given traffic
  • too many workers – some processes get swapped out, with a heavy overall performance penalty
  • non-optimal forking policy or MPM model
  • stalling application code (e.g. due to external calls)
  • server resources insufficient for the given traffic

Monitoring Apache processes CPU and RAM consumption

The most important server characteristics in relation to a web server is the amount of RAM. A web server should never swap because swapping out a process will make request processing latency unacceptable. So it is important to understand the maximum number of workers that fit the RAM of your server and to adjust the MaxClients (or MaxRequestWorkers) Apache configuration directive accordingly. In order to do this you should observe how much RAM is consumed by how many Apache processes under normal conditions. Divide the first by the second, and compare it to your total available physical memory to understand how many Apache processes you can have on this server. The same procedure should be applied to the CPU, as your web application may be CPU-intensive.

Knowing the maximum number of Apache workers per server will in turn give you some insight into the traffic you can serve by the server. You can use this information to anticipate future upgrades of your infrastructure.

Anturis Apache Monitor collects data about Apache processes’ aggregate CPU and memory consumption. It will also alert you if the values exceed the desired limits.

Best practices: http://cdn.swcdn.net/creative/v14.9/pdf/techtips/1202_SAM_BestPracticesApache_TT.pdf

Mysql

http://jonathanhui.com/mysql-monitoring-performance

https://github.com/shoma/mysqlmonitor/blob/master/src/mysqlstatus.py

Get to grips with the Basics

Fundamental Server Resources

Monitoring of the underlying server’s resources is the basis of MySQL database monitoring. Server CPU and memory usage, free disk space, free swap space, and network usage are all important metrics and the perfect starting point for troubleshooting in case of problems. Anturis will let you know in advance about any overloads of these resources, giving you the chance to prevent negative impact on database operations. You will also be able to spot usage trends over time so as to get insights on when you need to replace/upgrade the hardware serving the MySQL database.

MySQL Availability

The next obvious thing to know is MySQL availability. Failure of MySQL usually makes reliant applications completely inoperable. Anturis alerts you immediately in case MySQL itself or the underlying server goes down.

MySQL Error Log

MySQL error log contains all the errors occurring while MySQL is running as well as some context information, like stack traces. Anturis will watch the log file for error entries and keep you informed when it finds them.

Key MySQL Server Metrics

There are two ways in which MySQL metrics are useful to you: error detection and trending. In the first case you are notified of hazardous conditions that require your attention. Such conditions are defined by the thresholds on the selected metrics. In the second case, you have access to the metric graphs and you are able to troubleshoot by looking for unexpected changes and unusual patterns, and analyzing their reasons.

Here are a few examples of MySQL key performance metrics available with Anturis MySQL monitor.

  • Slow query rate. Slow queries are eating the precious resources of the server, leading to degraded performance and application failures. Having a slow query rate that is too high indicates that it is a good time to look into the slow query log and check what’s going on.
  • Connections usage. If your MySQL server reaches the maximum number of allowed connections the new client connections will simply be refused. Running out of connections signals an abnormal situation, such as when configuration changes may be needed.
  • InnoDB statistics. This includes InnoDB buffer pool usage, its’ miss and wait rates, and the InnoDB log cache wait rate. Using this data you are able to understand whether or not InnoDB performance is optimal.
  • MySQL query and key cache statistics. A low cache hit rate signifies that cache configuration is misaligned with application’s needs.

 

Tomcat

https://github.com/msacks/jythonJMXClient/blob/master/jythonJmxClient.py

 

Visualization

http://www.citronlab.com/programacion/javascript-graficos-con-morris-js/

Interactive Data Visualization using D3.js, DC.js, Nodejs and MongoDB

https://github.com/worldline/beekeeper

http://www.grepstar.org/iot-recipe-1-yun-mongodb-d3js/

http://adilmoujahid.com/posts/2015/01/interactive-data-visualization-d3-dc-python-mongodb/

https://github.com/StackStorm/collectd3

https://www.digitalocean.com/community/tutorials/an-introduction-to-tracking-statistics-with-graphite-statsd-and-collectd

Real Time

https://github.com/leggetter/socketalytics

Estándar

aprendiendo moodle

Antes de empezar, necesitamos saber que es moodle y su filosofía.

Moodle es un LMS (Learning Management System – Sistema de gestión de aprendizaje) opensource.

Permite que los educadores creen su propio espacio en un sitio web, donde proporcionar a los estudiantes cursos dinámicos y otros contenidos, de manera que el aprendizaje se extiende más allá de las aulas, pudiendo continuar en cualquier parte y a cualquier hora.

Continue Reading →

Estándar

workshop de iluminación de estudio jose mercado en madrid

Estamos organizando un workshop de iluminación en estudio en Madrid impartido por Jose Mercado los días 12 y 13 de Diciembre de 2014.

Hay 15 plazas disponibles y tendrá una duración de 12 horas repartidas entre los dos días.

2 horas dedicadas a la parte teórica, en la que se tratarán las bases de la iluminación:

  • Tipos de fuente de luz.
  • Ley inversa del cuadrado de la distancia.
  • Filosofía de iluminar.
  • Medición de luz.
  • Partes de un estudio y sus elementos.
  • Trato a los modelos.
  • Derechos de imagen
  • y muchos conceptos más.

10 horas para la parte práctica en la que trabajaremos con varios modelos y aprenderás a manejarte en un estudio desde la base.

El precio es de 150€.

Si te interesa debes mandar un mail a info@josemercadofotografia.com indicando:
nombre y apellidos
DNI
teléfono
email

 

MADRID2

 

Puedes descargar el FORMULARIO RESERVA CURSO ILUMINACIÓN Madrid con toda la información y los datos necesarios para realizar la reserva del curso.

Estándar

luna {1-52-2014}

La luna es la primera foto del proyecto 52 del 2014.
moon

En la cabalgata de 2014 de Pinto han sacado a pasear una luna luminosa y voladora, que mira como los niños recogen caramelos y ven pasar las  carrozas  de los Reyes Magos escoltados por los pajes.

slidedeck2 screenshot
Estándar

slidedeck2 slider wordpress plugin review

So you need an easy to set up image slider? here it is the solution SlideDeck 2.
SlideDeck 2 is a wordpress plugin wich allow you to create sliders using images, video, text, and even custom HTML.

It’s ideal to show your photos (find an example below) from different sources (instagram, flickr,…),
you can also use it to show your customers product features, make an online presentation,…

A cool way to spice up your wordpress website without extra programming.

In less than two minutes I’ve included the slider with my instagram photos, SlideDeck 2 managed everything including the authorization process with instagram.
Then I only had to choose how many photos I want to display and insert into my post using the wordpress post editor.

[SlideDeck2 id=3806]

SlideDeck 2 is simple, powerful and easy to customize.

In the example I had to customize the width of my slider because of the width of the container, you can choose from predefined widths or type your own dimensions.

slidedeck2 customization
slidedeck2 customization

The «lenses» are predefined themes for your slider, you can choose from a wide range of them, and if you need more
personalization you can program one to match your style.

slidedeck2 lenses
slidedeck2 lenses

In the following example I’ve used SlideDeck2 to show product features, with images and text.

[SlideDeck2 id=3820]

 

And the last example, combining my 500px and flickr photos.

[SlideDeck2 id=3826]

 

I you like it , get more info and pricing by clicking on the baner below.


SlideDeck.js | Beautiful jQuery sliders in minutes

Estándar

Clase de dominio Grails con las provincias españolas

Aquí dejo una clase de dominio de Grails con todas las provincias españolas.

No es que sea tecnología de la NASA, pero es un rollo escribirla, y ya que no la he encontrado ya hecha,

la comparto para vuestro uso y disfrute.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
class Provincia {
 
    String codigo
    String nombre
 
    static constraints = {
    }
 
    static void inicializar() {
    	if(!Provincia.findByCodigo('01')) {new Provincia(codigo:'01',nombre:'Álava').save(flush:true)}
    	if(!Provincia.findByCodigo('02')) {new Provincia(codigo:'02',nombre:'Albacete').save(flush:true)}
    	if(!Provincia.findByCodigo('03')) {new Provincia(codigo:'03',nombre:'Alicante').save(flush:true)}
    	if(!Provincia.findByCodigo('04')) {new Provincia(codigo:'04',nombre:'Almería').save(flush:true)}    	
    	if(!Provincia.findByCodigo('05')) {new Provincia(codigo:'05',nombre:'Ávila').save(flush:true)}
    	if(!Provincia.findByCodigo('06')) {new Provincia(codigo:'06',nombre:'Badajoz').save(flush:true)}
    	if(!Provincia.findByCodigo('07')) {new Provincia(codigo:'07',nombre:'Baleares').save(flush:true)}
    	if(!Provincia.findByCodigo('08')) {new Provincia(codigo:'08',nombre:'Barcelona').save(flush:true)}
    	if(!Provincia.findByCodigo('09')) {new Provincia(codigo:'09',nombre:'Burgos').save(flush:true)}
    	if(!Provincia.findByCodigo('10')) {new Provincia(codigo:'10',nombre:'Cáceres').save(flush:true)}
 
    	if(!Provincia.findByCodigo('11')) {new Provincia(codigo:'11',nombre:'Cádiz').save(flush:true)}
    	if(!Provincia.findByCodigo('12')) {new Provincia(codigo:'12',nombre:'Castellón').save(flush:true)}
    	if(!Provincia.findByCodigo('13')) {new Provincia(codigo:'13',nombre:'Ciudad Real').save(flush:true)}
    	if(!Provincia.findByCodigo('14')) {new Provincia(codigo:'14',nombre:'Córdoba').save(flush:true)}
    	if(!Provincia.findByCodigo('15')) {new Provincia(codigo:'15',nombre:'Coruña (La)').save(flush:true)}
    	if(!Provincia.findByCodigo('16')) {new Provincia(codigo:'16',nombre:'Cuenca').save(flush:true)}
    	if(!Provincia.findByCodigo('17')) {new Provincia(codigo:'17',nombre:'Gerona').save(flush:true)}
    	if(!Provincia.findByCodigo('18')) {new Provincia(codigo:'18',nombre:'Granada').save(flush:true)}
    	if(!Provincia.findByCodigo('19')) {new Provincia(codigo:'19',nombre:'Guadalajara').save(flush:true)}
    	if(!Provincia.findByCodigo('20')) {new Provincia(codigo:'20',nombre:'Guipuzcoa').save(flush:true)}
 
    	if(!Provincia.findByCodigo('21')) {new Provincia(codigo:'21',nombre:'Huelva').save(flush:true)}
    	if(!Provincia.findByCodigo('22')) {new Provincia(codigo:'22',nombre:'Huesca').save(flush:true)}
    	if(!Provincia.findByCodigo('23')) {new Provincia(codigo:'23',nombre:'Jaen').save(flush:true)}
    	if(!Provincia.findByCodigo('24')) {new Provincia(codigo:'24',nombre:'León').save(flush:true)}
    	if(!Provincia.findByCodigo('25')) {new Provincia(codigo:'25',nombre:'Lérida').save(flush:true)}
    	if(!Provincia.findByCodigo('26')) {new Provincia(codigo:'26',nombre:'Rioja (La)').save(flush:true)}
    	if(!Provincia.findByCodigo('27')) {new Provincia(codigo:'27',nombre:'Lugo').save(flush:true)}
    	if(!Provincia.findByCodigo('28')) {new Provincia(codigo:'28',nombre:'Madrid').save(flush:true)}
    	if(!Provincia.findByCodigo('29')) {new Provincia(codigo:'29',nombre:'Málaga').save(flush:true)}
    	if(!Provincia.findByCodigo('30')) {new Provincia(codigo:'30',nombre:'Murcia').save(flush:true)}
 
    	if(!Provincia.findByCodigo('31')) {new Provincia(codigo:'31',nombre:'Navarra').save(flush:true)}
    	if(!Provincia.findByCodigo('32')) {new Provincia(codigo:'32',nombre:'Orense').save(flush:true)}
    	if(!Provincia.findByCodigo('33')) {new Provincia(codigo:'33',nombre:'Asturias').save(flush:true)}
    	if(!Provincia.findByCodigo('34')) {new Provincia(codigo:'34',nombre:'Palencia').save(flush:true)}
    	if(!Provincia.findByCodigo('35')) {new Provincia(codigo:'35',nombre:'Palmas (Las)').save(flush:true)}
    	if(!Provincia.findByCodigo('36')) {new Provincia(codigo:'36',nombre:'Pontevedra').save(flush:true)}
    	if(!Provincia.findByCodigo('37')) {new Provincia(codigo:'37',nombre:'Salamanca').save(flush:true)}
    	if(!Provincia.findByCodigo('38')) {new Provincia(codigo:'38',nombre:'Tenerife (S.C.)').save(flush:true)}
    	if(!Provincia.findByCodigo('39')) {new Provincia(codigo:'39',nombre:'Cantabria').save(flush:true)}
    	if(!Provincia.findByCodigo('40')) {new Provincia(codigo:'40',nombre:'Segovia').save(flush:true)}
 
    	if(!Provincia.findByCodigo('41')) {new Provincia(codigo:'41',nombre:'Sevilla').save(flush:true)}
    	if(!Provincia.findByCodigo('42')) {new Provincia(codigo:'42',nombre:'Soria').save(flush:true)}
    	if(!Provincia.findByCodigo('43')) {new Provincia(codigo:'43',nombre:'Tarragona').save(flush:true)}
    	if(!Provincia.findByCodigo('44')) {new Provincia(codigo:'44',nombre:'Teruel').save(flush:true)}
    	if(!Provincia.findByCodigo('45')) {new Provincia(codigo:'45',nombre:'Toledo').save(flush:true)}
    	if(!Provincia.findByCodigo('46')) {new Provincia(codigo:'46',nombre:'Valencia').save(flush:true)}
    	if(!Provincia.findByCodigo('47')) {new Provincia(codigo:'47',nombre:'Valladolid').save(flush:true)}
    	if(!Provincia.findByCodigo('48')) {new Provincia(codigo:'48',nombre:'Vizcaya').save(flush:true)}
    	if(!Provincia.findByCodigo('49')) {new Provincia(codigo:'49',nombre:'Zamora').save(flush:true)}
    	if(!Provincia.findByCodigo('50')) {new Provincia(codigo:'50',nombre:'Zaragoza').save(flush:true)}
 
    	if(!Provincia.findByCodigo('51')) {new Provincia(codigo:'51',nombre:'Ceuta').save(flush:true)}
    	if(!Provincia.findByCodigo('52')) {new Provincia(codigo:'52',nombre:'Melilla').save(flush:true)}
 
    }
}

Para inicializar la tabla en la base de datos, yo la llamo en el BootStrap.groovy

 

1
Provincia.inicializar();

Espero que os sea útil

 

 

Estándar

Configuring Embedded Tomcat in Grails Development Environment

While you are developing a grails app, by default you are using an embedded Tomcat instance.

Sometimes you’ll need to do some configuration, in my case I had to proxy my tomcat instance with an Apache web server through mod_proxy_ajp .

When you have a standalone server running, you can easily change your server.xml file to add an AJP conector which listens for the Apache connections.

To do this within your grails app simply create a _Events.groovy file in your scripts folder of the grails project, and add the configurations needed by your app to the tomcat instance.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import grails.util.GrailsUtil
import org.apache.catalina.connector.Connector
import org.apache.coyote.http11.Http11AprProtocol
 
eventConfigureTomcat = {tomcat ->
if (GrailsUtil.environment == 'development')
{
def ajpConnector = new Connector("org.apache.coyote.ajp.AjpProtocol")
ajpConnector.port = 8009
ajpConnector.protocol = "AJP/1.3"
ajpConnector.redirectPort = 8443
ajpConnector.enableLookups = false
ajpConnector.setProperty("redirectPort", "8443")
ajpConnector.setProperty("protocol", "AJP/1.3")
ajpConnector.setProperty("enableLookups", "false")
tomcat.service.addConnector ajpConnector
println "Configured Tomcat ajp connector on port 8009"
}
}

You can also add other tomcat configuration such as Tomcat users and roles: link

 

 

 

Estándar

Instruct Groovy HTTPBuilder to handle a JSON response with wrong content type

While developing a utility script to monitor my mobile data consumed, I found a problem with

the response I was getting from the server. It content-type was set to ‘text/json’ instead of ‘application/json’, ‘application/javascript’ or ‘text/javascript’ wich are the content types used by HTTPBuilder to parse a response as JSON.

To solve this problem, one must add a new content type to the ParserRegistry and parse the response.

There are two ways to do it:

1
2
3
4
5
6
def httpBuilder = new HTTPBuilder(baseUrl)
 
httpBuilder.parser.'text/json' = { resp ->
   def bufferedText = resp.entity.content.getText( ParserRegistry.getCharset( resp ) ).trim()
   return new JsonSlurper().parseText( bufferedText )
}

The second ( reuse default )

1
2
def httpBuilder = new HTTPBuilder(baseUrl)
httpBuilder.parser.'text/json' = httpBuilder.parser.'application/json'

Now everything works again.

See you