NotworkManager and other small things

Had a few problems with system updates lately. One was an upgrade to my remaining slackware system that broke a few things. First it wanted to run LILO after updating the kernel and I said no (I don't use it); not sure if that would have run the grub setup but what happened it wasn't run. Fortunately one of the kernels in grub still existed and booted so it wasn't too hard to fix.

It also broke NetworkManager - or rather, it stopped working again. It's been a flakey piece of shit forever but I thought it was finally 'stable' enough to use (despite a few quirks on that machine like it not automatically reconnecting after waking up).

Well not so!

It simply wouldn't connect anymore. No idea why. I went back to using rc.inet1.conf and it now works flawlessly - even reconnects after waking up. I'd already done this (or equivalent) on all my other machines, and it seems to be with good reason.

Crackers

I knew the internet was pretty slimey these days but actually setting up a server on the naked internet over the last weekend was a bit of an eye opener.

I noticed a massive spike in traffic on the 15th - given that the only service running at the time was the 'experiment' page 1GB seemed a bit off. It was just someone brute-forcing sshd. Since this server went live on the 26th of march it has processed over 300 000 failed login attempts, I imagine (but haven't verified) most of those were on the 15th. They certainly weren't me.

It's probably just a drop in the ocean compared to all the `real' traffic but it seems such a waste. Yay for bots.

So i've put a few mitigations in place over the last few days:

• iptables rules to throttle new connections to port 22;
• disabled root login through ssh entirely;
• added a small blacklist using ipset.

I don't really want to have to maintain the last but i'll see how it goes.

Anyway it's sort of interesting to see the logins being used - root is obvious but hottie, mother and david don't seem too obvious.

Just for fun, here's the complete list of the usernames and frequency counts as of a few minutes ago.

       1 irc                1 sync               1 syslog             2             
       2 !                  2 12345678           2 1234qwer           2 123qwe      
       2 12qwaszx           2 1qazxsw2           2 654321             2 777777      
       2 aaron              2 abcd1234           2 admin@12           2 admintek    
       2 admUS              2 adriana            2 aion               2 alexis      
       2 amanda             2 amit               2 amy                2 andrea      
       2 angela             2 anthony            2 antiviru           2 ARGENTIN    
       2 arsenal            2 ashok              2 asshole            2 bananapi    
       2 bank               2 baseball           2 board              2 bobby       
       2 bonita             2 botmaste           2 byte               2 bytes       
       2 cameron            2 carditek           2 carmen             2 carolina    
       2 centos             2 chat               2 chelsea            2 chicken     
       2 chris              2 cinema             2 claudia            2 corazon     
       2 counters           2 crystal            2 cs                 2 csgoserv    
       2 csserver           2 customs            2 cuteako            2 cvs         
       2 cyber              2 data               2 db1                2 db2inst1    
       2 december           2 deploy             2 destiny            2 docker      
       2 download           2 dragon             2 dvd                2 edu         
       2 educatio           2 elastics           2 family             2 fedora      
       2 flower             2 forum              2 freedom            2 ftpuser1    
       2 gabriel            2 games              2 gaming             2 gb          
       2 ghost              2 gmodserv           2 gnu                2 gnuworld    
       2 greenday           2 harley             2 hdsf               2 hiitplc     
       2 home               2 hottie             2 html               2 http        
       2 hunter             2 idc!@              2 internet           2 ircd        
       2 isabel             2 jessica            2 jessie             2 jiamima     
       2 karen              2 kartel             2 keith              2 kernel      
       2 kitten             2 kmc                2 laura              2 lauren      
       2 libuuid            2 liferay            2 linaro             2 linux       
       2 linuxmin           2 liverpoo           2 logon              2 lovers      
       2 lpa                2 lucas              2 maganda            2 maggie      
       2 mail               2 mailman            2 maintain           2 manuel      
       2 marketin           2 matthew            2 mdb                2 miguel      
       2 muiehack           2 music              2 musicbot           2 mylove      
       2 myspace            2 nathan             2 Neuchate           2 Norwood     
       2 ns                 2 ns2                2 nuucp              2 october     
       2 odroid             2 openssh-           2 openvpn            2 oper        
       2 oracle2            2 orlando            2 otrs               2 pass        
       2 passw0rd           2 passwd             2 pc                 2 pepper      
       2 php                2 pictures           2 poohbear           2 portal      
       2 pretty             2 princess           2 proba              2 proftpd     
       2 project            2 p@ssw0rd           2 purple             2 q1w2e3r4    
       2 qazwsx             2 qwe123             2 qwerty             2 radio       
       2 rangers            2 rdp                2 redis              2 redmine     
       2 richard            2 root123            2 rootme             2 rsync       
       2 sakura             2 saw                2 scanner            2 security    
       2 servercs           2 serverpi           2 services           2 shell       
       2 sinus123           2 skan               2 skaner             2 snoopy      
       2 soccer             2 soft               2 software           2 steven      
       2 sweetie            2 sweety             2 tequiero           2 test123     
       2 test5              2 test6              2 testftp            2 tim         
       2 tomcat7            2 transfer           2 tsserver           2 ucpss       
       2 Untersee           2 upload             2 upport             2 uptime      
       2 user02             2 veronica           2 victor             2 video       
       2 virus              2 visitor            2 vnc                2 volumio     
       2 webconfi           2 webporta           2 webtest            2 Welcome1    
       2 wmware             2 x                  2 xbmc               2 xuelp123    
       2 zhaowei            2 zxin10             4 50cent             4 666666      
       4 admin123           4 alan               4 alarm              4 alejandr    
       4 alpine             4 andy               4 antonio            4 babygirl    
       4 bamboo             4 bin                4 blankend           4 build       
       4 carlos             4 control            4 csgo               4 daemon      
       4 daniela            4 dante              4 database           4 debian-s    
       4 dev                4 edi                4 fabricio           4 fabrizio    
       4 forever            4 gian               4 giorgio            4 giovanni    
       4 hannah             4 hello              4 iloveyou           4 jira        
       4 justin             4 leonardo           4 marco              4 mine        
       4 minecraf           4 naruto             4 nas                4 nginx       
       4 odoo               4 odoo2              4 oracle4            4 packer      
       4 patricia           4 patrizio           4 paul               4 plex        
       4 qwer1234           4 rebecca            4 roberto            4 rocco       
       4 sergio             4 shadow             4 shorty             4 shoutcas    
       4 staff              4 sysop              4 t7adm              4 test4       
       4 tsbot              4 vincenzi           4 vitaly             4 web         
       4 welcome            6 2Wire              6 admin2             6 amber       
       6 bot                6 camera             6 develope           6 dummy       
       6 Guest              6 hduser             6 jason              6 max         
       6 mobile             6 mythtv             6 netman             6 proxy       
       6 !root              6 Root               6 samba              6 server      
       6 sinus              6 temp               6 teste              6 training    
       6 ts3bot             6 ts3sleep           6 ts3user            6 vagrant     
       6 vps                6 zimeip             7 sys                8 albert      
       8 alessio            8 alex               8 anna               8 aurora      
       8 bianca             8 elena              8 enrica             8 ethos       
       8 hadoop             8 informix           8 lorenco            8 lorenzo     
       8 lucaluca           8 luigi              8 luka               8 marcel      
       8 marcello           8 maria              8 marta              8 massimo     
       8 mattia             8 olivia             8 oracle1            8 pia         
       8 piero              8 pippo              8 romeo              8 sinusbot    
       8 suporte            8 t7inst             8 test7              8 testing     
       8 tommaso            8 ts                 8 user3              8 valerio     
      10 0101              10 admins            10 cpanel            10 danny       
      10 dbuser            10 gnats             10 john              10 lavander    
      10 michael           10 miner             10 office            10 oracle3     
      10 postmast          10 prueba            10 test1             10 test8       
      10 tplink            10 user2             10 vmuser            12 101         
      12 123321            12 1502              12 266344            12 3comcso     
      12 aaa               12 acc               12 adam              12 adfexc      
      12 Admin             12 ADMN              12 agent             12 alessand    
      12 am                12 api               12 avahi             12 bill        
      12 bob               12 Cisco             12 draytek           12 echo        
      12 engineer          12 enrique           12 fax               12 gopher      
      12 helpdesk          12 houx              12 installe          12 kodi        
      12 luca              12 mario             12 mark              12 matteo      
      12 mike              12 mtch              12 naadmin           12 NAU         
      12 nt                12 pizza             12 Polycom           12 pos         
      12 print200          12 PRODDTA           12 PSEAdmin          12 radware     
      12 rapport           12 rcust             12 router            12 shop        
      12 steve             12 svin              12 svn               12 Sweex       
      12 SYSADM            12 SYSDBA            12 target            12 telco       
      12 telecom           12 ts3serve          12 ubadmin           12 user01      
      12 USERID            12 username          12 vcr               12 vmadmin     
      12 VNC               12 volition          12 vt100             12 VTech       
      12 webadmin          14 1111              14 a                 14 demo        
      14 ftptest           14 info              14 library           14 media       
      14 midgear           14 superman          14 system            14 www-data    
      16 angelo            16 cvsuser           16 cyrus             16 donatell    
      16 dvs               16 firebird          16 oracle5           16 scan        
      16 supervis          16 vyatta            18 Administ          18 backup      
      18 ftpadmin          18 git               18 jenkins           18 mtcl        
      18 raspberr          18 steam             18 teamspea          18 tech        
      18 ts3               18 User              18 www               20 debian      
      20 martin            20 sales             20 sshd              20 test9       
      22 12345             22 oliver            22 setup             22 telecoma    
      22 test2             24 123456            24 client            24 daniel      
      24 Operator          24 student           24 sysadm            26 0           
      26 backuppc          26 vision            28 avis              28 cisco       
      28 david             28 Manageme          28 mother            28 mysql       
      28 sysadmin          28 uucp              30 plcmspip          30 public      
      32 apache            32 master            34 applmgr           34 osmc        
      34 phion             36 butter            36 squid             38 111111      
      38 cacti             38 cron              38 nobody            38 user1       
      38 wp-user           38 zimbra            40 scaner            42 anonymou    
      42 castis            42 ftp_user          46 123               46 22          
      46 PlcmSpIp          46 usuario           46 webmaste          50 monitor     
      54 qhsuppor          54 testuser          60 manager           60 sybase      
      62 jboss             64 ftp_test          65 service           72 tomcat      
      76 zabbix            78 administ          78 super             90 default     
      96 adm               96 nagios           102 1234             112 operator    
     128 oracle           130 postgres         142 ftp              228 ftpuser     
     242 support          292 pi              4140 ubuntu          4192 guest       
    4268 ubnt            4302 test            4434 user            6081 admin       
  267194 root

Given this i'm not entirely sure it's a great idea to be running cvstrac - it appears to be unmaintained and so on, but it's only intended to be a short-term solution anyway.

Weather's too nice to be inside, i've done enough hours for the week, and a brother is in town so I think it's beer time!

Update 22/4/18: Thinking about the strange usernames, they are probably bot related accounts? Doesn't really matter.

Welcome to the ZedZone

First post on the new blog!

Experimenting with a very rudimentary partly manual, somewhat temporary posting mechanism until I can sort something better out.

Apart from setting this up i've done a little hardening on the software and the system. I tuned ssh a little bit. I added robots.txt to code.zedzone.au to stop indexers creating potentially infinite references. And I changed the blog indexing method and some of the url's for the same reason.

Mostly root login attempts via ssh, some looking for various (mostly php) server stuff (which isn't installed), and google bot getting a bit cross eyed at some of my url alias/rewriting/cgi mistakes.

time to put some money where my mouth is?

Due to some of the surveillance/tracking stuff in the news of late I went down a bit of a rabbit hole of looking at various 'secure' email and so on to do something about my gmail account and blogger.

I didn't really find anything I liked so instead i got myself a domain and a virtual private server to play with. I had originally intended to set something up on my home network via a static address but I never got around to it. Unfortunately getting a vps in switzerland is cheaper than even doing that - and that's with the virtual hardware and unlimited network bandwidth the latter of which is big buckeroonies in australia.

Well I still haven't done anything with it apart from setup apache with a nothing homepage pointing back to my isp hosted stuff, but it's at www.zedzone.au.

I'm still pretty lazy on the whole idea so it might take a while but i'll move my code and blog and eventually add an email to it. I don't particularly like working at that end these days (despite multiple shots at CMS subsystems) - too much security and complexity to worry about - but i'll see how it goes.

Although logging in here to post this did give me pause for thought on getting a bit less lazy about it. Despite never having offering it before blogger just gave me a page to buy and link a custom domain for a-hackers-craic.

Thanks google for once again mining my email to sell me something I already bought!

Next it'll be combat boots, kilts, and beer ... Adelaide Review article which includes bits about me (not sure why i didn't introduce myself as Zed that time, would've been better!).

Update: 15/4/2018. I finally got off my laze arse and spent a wet afternoon playing with the site - i've copied my internode (ISP) pages to the server, fixed some paths, and installed cvstrac for browsing the source (of some of the projects) via the new virtual host code.zedzone.au.

Still to do is the blog.

intel bugs, surveys, random shit

Well i've been following the whole "spectre" and "meltdown" saga over the last week or so. Apart from the pretty offensive development of branding (ugh) for a set of mostly unrelated bugs, loads of pro-intel FUD, misdirection and so on, it's been a bit of a fun ride of stupidity.

I haven't seen any patches for my laptop or work computers so far - one because i don't run any av software on microsoft? Ugh, what a fucking mess. I barely use that machine any more anyway, even less so while running that junk.

What's most appalling is that the problem has been sat on for 6 months under "responsible disclosure" while intel continued selling gimped hardware based on bogus benchmarks. And yet the linux and *bsd developers are either only just starting on or working on cleaning up fixes very recently - even intel is still fucking around with microcode updates.

The tech press, apart from The Register (who really broke this story) has been pretty shithouse too. "big nothingburger" for a massive massive security hole that isn't actually fixed by any of these OS kernel patches, merely mitigated. It's a big big serious problem, with big big serious costs, and the handling the disclosure has been an utter disaster - seemingly designed to spread fud about the impacts and costs. Still, if you relied on a single supplier - and particularly a nefarious piece of shit company like Intel - for any serious hardware investment, sucked in I guess? We all have to pay for your stupidity in the end regardless.

Blah.

I was insomniac[sic] late night and bored so I took the latest stack overflow survey. Apart from some pretty strange questions it's pretty clear why nobody reads my blog - no mention of almost any of the technologies I use day to day. Well at least they had the brains to identify C and C++ as two separate languages I suppose.

Some of the "order these in importance to you" questions I just skipped - I didn't care about the topic at all. A minor problem was a bit of confusion with "framework" "language", etc - where would OpenCL fit if they even included it? No CUDA either for that matter. Plenty of questions about advertising. No I don't use an ad-blocker but I turn off javascript which makes the site oh so much faster and kills most adverts as a bonus.

Sleep has been pretty miserable lately - probably a side-effect of overdoing it a bit, but I think i've gained enough weight again for the sleep apnoea to kick in. One example of overdoing it was the last weekend. Hit the beach Saturday afternoon, dropped in at a mates place - drank a carton of (bloody expensive) piss, went to a party (total strangers apart from him), was up all night whilst everyone else was falling asleep, back to his place to pick my bike up - for a couple more, another dip, and finally dropped by the pub on the way home for a fast (very slow!) one. Barmaid thought i was managing well all things considered but maybe that's because I always look exhausted and feel like shit? Nothing like a 36 hour day to get you tired; and then I was awake after only 4 hours of bloody sleep. For a couple of silly reasons I also got sunburnt to a crisp, oops.

Finally kicking into summer mode!

Another year down

Well that was 2017 I guess. Better than 2016 for me at least.

Pretty much recovered from NYE and NYEE drinking but I might take it easy for a couple of days. Waiting for the weather to heat up enough to get back to the beach, did some gardening. The replacement BIOS arrived today so I resurrected my PC too - still seems a bit funny so i'll probably get another one soonish but it works again for now.

I found a laser printer dumped on the side of the road so took it apart for something to do. Surprising amount of screws, custom springs, cogs, and lots and lots of tough plastic. Probably nothing I can do with it but the imaging unit is kinda cool - rotating mirror, a couple of lenses and mirrors. I suppose the more surprising thing is how much technology is placed in what is essentially a throwaway, one-use device. The "consumable" isn't the toner in these things, it's the whole machine; and what's worse is the realisation that the economies of scale mean it couldn't be done any other way. The world is so fucked.

When I wrote the following on the new jjmpeg home page:

Version 3+ is a complete rewrite from earlier versions which have effectively dissapeared from the internet after google code closed down.

I didn't realise just how true it was. Without google code and regular blog updates my projects have basically vanished from the internets - and more so from google search than others. I'm not particularly surprised there is no real interest in the projects themselves but effectively vanishing is a bit weird. Unless you're using some proprietary publishing platform you essentially don't exist.

Well i've got a few weeks off at least anyway; i'll take a break from the computer, watch some cricket, and basically just bum around a bit.

jjmpeg 3.0 released

Put enough together to push out a release of jjmpeg.

It ended up 1700 lines of Java, 2000 lines of C, and 300 lines of Perl.

Apart from supporting the latest version of FFmpeg (at least when I started a couple of weeks ago), it's smaller, cleaner, and more complete than any previous version. Having said that this is essentially just a beta release.

This one is licensed GNU General Public License Version 3 (or later).

I've kinda had enough for the moment so it's a pretty bare home page, but it's there.

Merry XMAS!

damned enums

Been a long week but i'm finally done with work for another year. Although it's mostly a long week because of the late nights working on jjmpeg ...

One of the things I did was fill out/sync up the important enums - AVCodecID, AVPixelFormat, AVSampleFormat, and so on. Previously the pixel format and sample formats were also Java enums - which can be convenient at times and provides some more (albeit much much overvalued) 'type safety'.

This was fairly easy because the PixelFormat was a simple densely ordered C enum so i could map between the two with a simple +-1. Unfortunately someone decided to add a big hole in the middle of it sometime between 0.10 and 3.4, ... so I gave up and just converted it to a class holding static final int's, and to make it consistent I did that with the other enumerations as well. It doesn't really make the classes any harder to use and improves the class size and memory footprint. I just added some methods to access libav*'s metadata information so I can still map between string representations and so on.

I had to add a small compilation stage which extracts the enums from the header files and converts them to a C file which when compiled and run produces the Java source ... this seemed the absolute shortest path to ensuring I got accurate numbers based on the ffmpeg build configuration.

So after about a weeks worth of solid work it's grown somewhat (about 2KLOC Java, and 2KLOC C, counting lines with ";{}") and the TODO list is getting pretty short.

I would like to clean up the exception design a bit - unfortunately i'm just not very good at that (who is?) but i'd like to get better. The build system is clean and simple but could be improved and needs to include the aforementioned enum stuff, a dist target and versioning. Logging would be nice (both redirect ffmpeg to java.util.logging and some for jjmpeg itself). JJMediaWriter? Fix the license headers, add at least a README.

Not today though, today I drink.

jjmpeg, jni, javafx

So I guess the mood took me, I somehow ended poking away until the very late morning hours (4am) the last couple of nights hacking on jjmpeg. Just one more small problem to solve ... that never ended. Today I should've been working but i've given up and will write it off, it's nearly xmas break anyway so there's no rush, and i'm ahead of the curve anyway.

JJMediaReader

I got this ported over and playing video fairly easily, and then went through on a cleanup spree. I removed all the BufferedImage, multi-buffering, and scaling stuff and a few other experiments which never worked. Some api changes allowed me to consolidate more code into a base class, and some changes to AVStream necessitated a different approach to initialising the AVCodecContext (using AVCodecParameters). I made a few other little tweaks on the way.

The reason I removed the BufferedImage code is because I didn't want to pollute it with "platform specific" code. i.e. swing, javafx, etc. I've moved that functionality into a separate namespace (module?).

My first cut just took the BufferedImage code and put it into another class which provides the functionality by taking the current AVFrame from the JJMediaReader video stream. This'll probably do but when working on similar functionality for JavaFX I took a completely different approach - implementing a native PixelReader() so that the native code can decide the best way to write to the buffer. This is perhaps a little more work but is a lot cleaner to use.

swscale

jjmpeg1 lets you scale images 'directly' to/from primitive arrays or direct ByteBuffers in addition to AVFrame. Since they have no structure description (size, format), this either has to be passed in to the functions (messy) or stored in the object (also messy). jjmpeg1 used the latter option and for now I simply haven't implemented them.

The PixelReader mentioned above does implement it internally but for code re-use it might make sense to implement them with the structure information as explicit parameters, and use higher level objects such as PixelReader/Writer to track such information. On the other hand the native code has access to more information so it also makes sense to leave it there.

I went a bit further and created a re-usable super-class that does most of the work and toolkit specific routines only have to tweak the invocation. This approach hides libswscale behind another api. The slice conversions don't work properly but they're not necessary.

jni

So far I had public constructors and `finalisers' because otherwise the reflection code failed. That's a bit too ugly (and `dangerous') so I made them private. The reflection code just had to look up the methods and set them Accessible.

    Constructor cc = jtype.getDeclaredConstructor(Long.TYPE);

    cc.setAccessible(true);

    return cc.newInstance(p);

Whilst working on JJMediaReader I hit a snag with the issue of ownership. In most cases objects are either created anew and released (or gc'd) by the Java code, or are simply references to data managed elsewhere. I was addressing the latter problem by simply having an empty release() method for the instance, but that isn't flexible enough because some objects are created or referenced the the context determines which.

So I expanded the Java-side object tracking to include a `refer' method in addition to the 'resolve' method. `resolve' either creates a new instance or returns and existing one with a weak-reference object which will invoke the static release method when it gets finalised. `refer' on the other hand does the same thing but uses a different weak-reference object which does nothing.

I then noticed (the rather obvious) that if an object is created, it can't possibly 'go away' from the object tracking if it is still alive; therefore the `resolve' method was doing redundant work. So I created another `create' method which assumes the object is always a new one and simply adds it to the table. It can also do some checking but i'm pretty sure it can't fail ...

If on the other hand the underlying data was reference counted then the `resolve' method would be useful since it would be possible to lookup an existing object despite it being `released'. So i'll keep it in CObject.

As part of this change I also improved CObject in other ways.

I was storing the weak reference to the object itself inside the object so I could implement explicit release and to avoid copying the pointer. I removed that reference and only store the pointer now. The WeakReference it already tracked in a hash table so I just look it up if I need it. This lets me change the jni code to use a field lookup rather than a function call to retrieve it (I doubt it makes much perf difference but I will profile it at some point).

I also had some pretty messy "cross-layer" use of static variables and messy synchronisation code. I moved all map references to outside of the weak reference routine and use a synchronised map for the pointer to object table.

For explicit release I simply call .clear() and .enqueue() on the WeakReference - which seems to do the right thing, and simplifies the release code (at least conceptually) since it always runs on the same thread.