Release Notes

v3.6.2 patches

  1. batch-patch-29-02-2016
  2. ssl-cert-handling
  3. no-response-from-restarted-instance
  4. add-proxy-settings-to-configs
  5. disk-quota-cve-2016-0780
  6. ssl-cert-fixes
  7. hotfix-revert-3036
  8. polipo-bypass
  9. delete-token
  10. fix-token-auth
  11. drop-v1-app-limit
  12. 012-batch-patch-05-07-2016
  13. 013-ldap-null-user
  14. 014-windows-role-on-linux
  15. 015-kato-relocate
  16. 016-kato-attach-copy-cert
  17. 017-supervisord-configurable-password
  18. 018-websocket-affinity
  19. 019-faster-app-deployment
  20. 020-aok-token-expiry
  21. 021-kato-info-remote-node
  22. 022-buildpack-import-skip
  23. 023-ldap-broken-on-delete-user
  24. 024-fix-container-cron

Helion Stackato CLI Client 3.2.6

Helion Stackato CLI Client 3.2.5

  • Download v3.2.5: Administrators should update the client downloads in the Management Console.
  • [APAAS-3019]: Fixed returning failure on successful application push
    • changed abort condition to 'all flapping'
    • excluding 'down' from the abort condition
    • rephrased error for clarity
    • extended timeout on this to 30 seconds.
    • generating exit code 2 on this timeout to signal the unsureness of the application's state up to the client's caller.
  • [APAAS-3055]: Fixed mishandling of TLS-specific -servername option triggered by "no_proxy" match.
  • [APAAS-2854]: Ensured that running a regular setup blocks future setup-from-config, preventing a double setup.

v3.6.2 (Feb. 29, 2016)

New Features

Patch Roll-Up

All patches released since v3.6.1 have been included with this release.

  • docker-app-log: Fixes logging for some Docker images.

  • routes-flatlined: Ensures that any route but the last one is removed after flatlining.

  • internal-docker-registry-upgrade: Allows Helion Stackato to perform remote registry pulls from Docker Hub v2.

  • docker-version-upgrade: Upgrades the Docker client shipped in Helion Stackato to v1.8.3.

  • config-redis-handle-retry: Improves database connection attempts while Redis is loading.

  • batch-patch-21-12-2015: This batch patch addresses the following issues:

    • The application store fails to load if it uses HTTPS.
    • The kato op regenerate all command exits while regenerating the encryption key for the Cloud Controller database.
    • The stackato files <docker-app-name> command does not run.
    • Routes that use uppercase characters (for example, do not work.
    • Administrative users created by membership in LDAP Admin Groups do not have the same permissions as users created by built-in user management.
    • The Cloud Controller issues multiple start messages, causing the Health Manager to terminate too many instances.
    • The MySQL service user runs as the stackato user, giving the process elevated access privileges.
    • stackato_rest crashes when config-redis is restarted.
    • Helion Stackato does not report DEA memory usage correctly.
    • The Helion Stackato REST API hangs as a result of ending TCP connections prematurely. The REST API requires a manual restart of the controller to resume regular operations.

    In addition, this patch removes the kato node reset factory command.

  • batch-patch-13-01-2016: This batch patch performs the following actions:

    • Upgrades Futuroscope
    • Allows Helion Stackato to store application versions according to the set number of droplets per application
    • Allows Fence to get the correct container memory usage from Docker images
    • Adds the MySQL service user
  • pip-dependencies-installation: Upgrades the version of setuptools in the Helion Stackato base image to v19.2.

  • java-cve: Fixes the CVE-2016-0708 critical vulnerability by removing the appropriate software hooks that can compromise your files.

  • stack-cflinuxfs2: Adds support for installing the CF Linux FS2 stack and other stacks.

  • windea: Adds support for adding a Windows DEA node to a Helion Stackato cluster.

  • add-usb: Adds support for connecting services to a Helion Stackato cluster using the Universal Service Broker.

Helion Stackato CLI Client 3.2.4

  • [2718] Fixed the early abort on zero-downtime switchover for Docker applications.
  • [2690] Extended the description of the option --disk for the push command.
  • Added support for MS-SQL as a tunnel-able databases.
  • [302164] Modified manifest processing and unified the key disk and the new cf key 'disk_quota'. If both keys are present, disk_quota takes priority over disk.
  • [302133] Added support for user-specified service tags and support for arbitrary service parameters.
  • [302094] Changed the error message App has not finished staging to App package invalid, no bits uploaded.
  • [302093] Modified the open command to reject stopped applications.
  • [302091] Modified the dbshell command to distinguish between an existing but unbound service and a missing service within generated error messages.
  • [302090] Modified the apps command, adding the new Disk column and showing additional data with unit letters for Mem and Disk.
  • Improved the messaging of (set-)environment-variable-group commands.
  • Reworked the messaging of (un)set-placement-zone commands.
  • Extended regular quotas using the instance memory attribute.
  • Extended the display of application details to show the application stack, and the time of last upload.
  • [301838] Added the push --no-urls option and (and the cf-compatible alias --no-routes, allowing worker apps to be specified from the command line.
  • [104727] Added support for the options -v, --version, -h, --help, and -? and implemented the options as commands mapping to version and help.
  • [104423] Reduced the use of inline-relation-depth > 2 to the new maximum of 2.

v3.6.1 (Nov. 1, 2015)

New Branding

  • Stackato is now HPE Helion Stackato: Version 3.6.1 marks the first official release of Stackato as an HPE Helion product.
  • Look and Feel: Numerous updates to the web management console.
    • The App Store has been renamed to Sample Applications.

Patch Roll-Up

All patches released since v3.6.0 have been included with this release.

  • cc-regex-fix: Clean up CC regex.
  • dea-app-log-fix: Fixes Docker apps missing apptail logs.
  • dea-role-ssh-fix: Resolves issue of restarting DEA prompting for admin password on other DEA nodes.
  • dbshell-command-fix: Fixes stackato dbshell command.
  • ssl-cert-copy-ssh-creds: makes scp call as stackato user instead of root.
  • cumulative-patch-1:
    • Minor issue with authentication component failing to re-advertise itself if it crashed for a length of time
    • Issue with importing data from previous version if the import included admin buildpacks
    • Removes requirement to authenticate newly attached DEAs with every existing DEA. Also removes this requirement from restarting the DEA role or restarting the DEA itself.
    • Allows specifying source using https for custom appstore URLs in the web console.
  • appstore-url-fix: Update appstore URLs to 3.6.
  • detached-node-patch-state-fix: Fixes detached nodes having their patch state reset.
  • services-list-description: Add default services description of N/A.
  • dea-add-ssh-fix: Removes automated ssh-copy-id from starting DEA role.
  • memcached-url-fix: Fixes MEMCACHED_URL environment variable when using a memcached service.
  • router-keepalive: Fix router keep-alive header issues.
  • aok-readvertise: Improve AOK failure issues with router.
  • web-console-cluster-settings-whitelist: Hides the settings button for nodes that do not contain a base node.
  • redis-vulnerability-fix: Fixes the Redis EVAL Lua sandbox escape.
  • manually-remove-container: Do not use docker run --rm to read the stackon manifest.
  • sso-internal-server-error-fix: Fixes internal server error happening in some cases when using SSO apps.
  • buildpack-import-fix: Fixed duplicate key value violates unique constraint "buildpacks_pkey"
  • ifconfig-parsing: Fixes app timeout and fail on on DEAs with malformed or unusual network device names.

Helion Stackato CLI Client 3.2.3

  • [302380] Fixed missing update of in-memory auth database when refreshing a token.
  • [302250] Fix missing update of environment variables when properly changed (--env-mode replace, or new variables under --reset).
  • [302015] Fixed display of TclOO obj command and CF uuid for stack during push, now showing the proper stack names for old and new. Further ensured that the labels are shown in lower case and dynamically vertically aligned, without superfluous spaces after the label (and new value).
  • [301950] Modified download of buildpack to accept any content-type.
  • [301929] Fixed incorrect use of 'dict set'.
  • [301886] Fix mishandling of trailing / in directory name given to "create-buildpack" causing use of file name consisting of only an extension, and breaking the upload.
  • [301818] Change handling of 404 responses caused by the "files" command, show "No such file or directory" instead of an internal error. This could happen when trying to escape the application top directory.

v3.6.0 (May 8, 2015)

New Features

  • Docker Apps: Deploy Docker images as apps in Stackato
    • Users deploy Docker images by specifying the image name or path with the stackato CLI client
    • Administrators control which users can deploy Docker apps, or which registries and images are allowed
    • Local caching registry servers in Stackato share Docker image layers to improve deployment efficiency
  • Cloud Foundry Feature Flags: Permission controls for organization Managers and Developers governing application upload and scaling, and the creation of domains, routes, service instances, and organizations.

Enhancements & Updates

  • [301156] Merge from upstream Cloud Foundry (v205)
  • [300275] Updated bundled buildpacks to latest released versions
  • [300131] Updated Ruby to 2.1.5
  • [301313] Update Docker to 1.5

Bug Fixes

  • [300155] Incorrect "patches available" notification in web console
  • [300048] Session cookies for web console
  • [105229] Add view for individual quota plans
  • [105318] Add a level of granularity to kato data export so it exports only the users/orgs portion of the cc database
  • [301489] malformed MEMCACHED_URL
  • [301471] Bad description of stack "lucid64"
  • [105896] Running 'kato op static_ip', hangs at 'Restarting Networking service'
  • [106134] Console Login: {"error": "bad_request", "error_description":"Incorrect_redirect_uri"}
  • [106323] REST-API access through Content-Cache
  • [300049] Passwords leaking in to log files
  • [300128] stackato buildpacks lost after kato data import
  • [300155] "80 patches available" on fresh 3.4.2 install
  • [300178] Allow renaming logs.* endpoint
  • [300195] stackato-release based builds
  • [300208] Kato::UI.puts outputs text when Kato::UI.disable set
  • [300350] Check for nulls in grub recipe
  • [300820] Upgrade MongoDB from 2.4 to 2.6
  • [301147] Add support for arbitrary version names to sentinel
  • [301240] Add support for basic auth to recipe/manifest downloading
  • [301242] Update sentinel to handle apt-cacher idempotently
  • [301247] Space -> Quota Usage should show "Org Quota Definition" in table when no space quota selected
  • [301297] WinDEA: Prevent web console adding additional roles to a windea node
  • [301301] Use global URL redirection instead of manually rewriting individual files
  • [301304] 'kato node upgrade' --download-only option no longer works.
  • [301319] Daemon should preserve folder structure for remote downloads
  • [301324] Stackato version not updated after upgrade
  • [301354] Update Sentinel to specify that a recipe should always be installed on upgrade
  • [301395] Create a 'version setting' recipe that sets Stackato versions in all required places at the very end of an upgrade
  • [301366] Sentinel needs to download images for intermediate versions up front
  • [301402] App 404 page flashes briefly after app deletion
  • [301403] Make patches UI more robust to prevent incorrect # of patches in display
  • [301428] Factory reset should delete fetched Docker images
  • [301452] apptail broken after upgrade
  • [301467] pre-running hooks and ssh/cron jobs do not see environment changes from ~/.profile.d

Stackato CLI Client 3.2.1

  • [301584] Force --cafile and --skip-ssl-validation defaults when cmdr triggers target access before argument completion phase, to have certs and skip/debug flags ready.

  • [301559, 301224, 300024] Process option --docker-image in the interactive SyncV2 code path.

  • More service information: state of last operation ... Added support, and display.

  • New undocumented command "debug-options" to list the set of --debug options available to developers.

  • Implemented support for environment-variable-groups (staging and running). Two new commands, "environment-variable-group" and "set-environment-variable-group".

  • Tweaks to the output of various commands generating listings, for more consistency.

  • Test suite updates.

  • Translate websocket issues on open into regular errors instead of having them as internal error. Extended websocket debugging. Activate deeper websocket package debug for --debug mgr|ws|data.

  • Modified internal http package to have the url which caused error on open in the message.

  • [301493] Memory reporting by the usage command, assume unit [Byte] for v2 API, stackato 3.x, usage key.

  • [301411] Gracefully handle the two missing attributes (overridden, default_value) CF dropped from the feature-flag entity spec.

  • [301346] Update internals to the new location for orphan-relations in list results. Key has changed from "relations" to "orphans". Client now supports both, for compatibility.

  • [301254] Gracefully degrade the listing of service plans when we do not have the permissions to see service-broker information.

  • [300462] Modified retrieval of route information to prefer direct access from the space over the getting the full list and client-side filtering, for proper fast knowledge of unused routes. Also fixed the same issue in the "delete-route" command.

  • [300024] Implemented a new option "--docker-image" for command "push" for the specification of docker images as applications.

  • Made "scp" more robust against the possibility of trying to access an instance which is down or flapping.

  • [104102] Restart tunnel app after binding a service, to make it fully known. Without restart the app will not see the new service and thus not allow connection.

  • [106212] Modified the handling of --path option to push command. Now has precedence of a manifest path: setting. Fixes the issue of a manifest.yml suppressing the use of --path, even through a default path:.

  • [105017] Added support for killing (== restarting) a specific application instance. The new command is actually an option: "restart --instance N".

  • [105944] Extended "target" command to allow for a --timeout declaration, and making it sticky. The --timeout of "push" now checks for such a sticky value.

  • [105550] Activated TLS 1.1 and 1.2 protocols for the client.

  • Tweaked error reporting involving an application rollback. Error is now reported before the rollback, the rollback is done, and the error is reported again, with a "Reminder" tag and that it forced the rollback.

  • [106151, 105886] Reworked handling of pre-push hooks to be more portable (SHELL, COMSPEC, /bin/sh fallback for unix(-like) environments).

  • [300114] Tweaked output of "apps" command to not show the columns "Restart" and "Drains" by default, to make it easier to show on narrow terminals. The old and very wide look can be restored via the new option "--full".

  • Extended the logstream display to recognize a few fixed strings in the texts and highlight them via colorization (warning, error, note).

  • [106086] Start log-stream for zero-downtime switchover as well, and use timeouts to detect its semi-start and semi-end.

  • [105916] Fixed spelling mistakes and inconsistencies of "buildpack".

  • [105899] Extended "target" command to allow for --no-ws|--ws declaration, and making them sticky. The --no-ws of push now checks for such a sticky flag.

  • [OTRS 030026186]: Fix un|bind-service command when talking to the "tunnel" application on a Stackato v2 target. Application names are strings, must not be mistaken for object. For S3 targets we additional check the class of the object.

  • [104626] Only for Windows, added a 1s delay between closing the temp file used by command "create-buildpack" and its deletion. Done to avoid the OS race where we try to delete the file while it is not yet unlocked. Small tweaks to the progress reporting.

  • [105748] Suppress SSL warnings for quiet commands like ssh, scp, etc., and when --json is given, to prevent interference with scripted use of this data.

  • [104652] Drop ".exe" extension from the application name shown in the help.

  • [104535] Extended the command "create-service-broker" with option "--public". The plans of the newly registered broker are immediately made public.

  • New command "show-service-broker" to show the information of a single named broker.

  • Extended command "service-plans" (and "services") to show the broker a plan is associated with, if any.

  • [104765, 104857] Extended the 'env' command to show the system variables as well.

  • [104693] Restrict (interactive/automatic) choice of org on login or other places to the set of orgs the currently logged in user belongs to, instead of allowing all.

  • [104294] Added alias name --placementzone for option --placement-zone of command push. Extended the manifest handling to recognize the key placementzone as a variant of the canonical key placement-zone.


    If both forms are specified, the canonical placement-zone takes precedence.

  • [300496] Limit inline-relation-depth to 2 for non-Stackato targets.

v3.4.2 (Dec. 17, 2014)

New Features

  • [103548] Apps can now be moved between Spaces (by Org Managers) and Orgs (by Admins).
  • Management Console UI for:
    • [102890] Quota management
    • User Provided Service Instances
    • Routes within a Space
    • All Routes (Admin view)
    • Space Quota Plans
    • Feature Flags
    • Toggling Admin/Non-admin view

Enhancements & Updates

  • [104831] Merge from upstream Cloud Foundry (August 2014)
  • [105462] Updated built-in Buildpacks
  • [104659] The Logyard stream can now be branded
  • [105465] Updated Node.js to v0.10.33
  • [105411] Update Docker to 1.3

Bug Fixes

Management Console

  • [102951] Consistent styling of common UI patterns
  • [102489] Show problem components in red
  • [102316] App logs in chronological order (previously reverse chronological)
  • [105420] Fixed usage reporting in Space quota view
  • [105413] Hide non-functional UI elements for users without permission to use them
  • [105352] START button disappears from web console in certain condition
  • Deleting an app now asks if you also want to delete the routes associated with that app
  • [104841] Router Stats Dashboard 'Connections' display expands outside the box when it is over 6 digits
  • [104866] JS error: Cannot read property 'trim' of undefined welcome.js.src.js:106
  • [104820, 104819, 104818] App file browser view bugs
  • [104809] Server date/time warning
  • [104028] Add "install app from app store" to the space page
  • [104703] Unexpected results from clicking on domain rows


  • [105318] Options for kato data export for exporting just the Cloud Controller main database (users, orgs, spaces).
  • [98559] kato status now faster. Contacts all nodes concurrently.
  • [105286] kato patch checks that it is up-to-date before applying patches.
  • [105467] kato start controller needs to check for multiple controllers
  • [105347] kato patch install attempts to repeatedly update other nodes
  • [105298] Kato emits "error getting sizes" message every 10 seconds
  • [105260] Installed patch status lost
  • [105014] Fixed recursion in kato patch update
  • [104860] kato patch install on non-existent patch should return proper error message
  • [104458] kato data import/export unauthorized service offering error
  • [104893] Inconsistent kato command kato data repair_routes
  • [104785] kato relocate containers does not maintain all DOCKER_OPTS
  • [104621] kato relocate failure: error trying to unmount an AUFS docker node
  • [104133] kato relocate does not work if data is on different device
  • [103770] 'kato node reset factory' fails with error

Docker & Fence

  • [105194] Fence crashing with fork error
  • [105123] Fence process keeps restarting
  • [105116] Fence connection issues
  • [105322, 105202, 105362] Clean up orphaned containers
  • [105295] app logs going to Fence
  • [105259] Fence crash on services node during patching


  • [104975, 105424] Upgrade issue with filesystem service
  • [104935] Random hang during upgrade
  • [104925] Fatal error in Bundler during upgrade
  • [104912] Pin bundler to a known good version in sentinel
  • [105698] Upgrade fails on stackato-tools-3.4.1 with 'no such directory'
  • [105690] Upgrade fails on client due to incorrect versions
  • [105689] sentinel download --for-upgrade should download intermediate versions
  • [104982] Failing with exception during mysql upgrade
  • [104842] Unable to delete app after upgrade from 3.2.1 -> 3.4.1
  • [104996, 104993] Log drains stop working after upgrade
  • [104554] Duplicate key value on first kato op remap_hosts after data import
  • [104499] 3.2.1->3.4.1 upgrade fails with NoMethodError


  • Various fixes for POODLE
  • [105437] stackato-ssh breaks when SSLv3 disabled
  • [105161] CVE-2014-5119 glibc vulnerability that leads to escalation


  • [105068] DEA lockup
  • [105514] Missing /usr/share/doc directory restored
  • [105513] run_fibered_command: Do not create /dev/fd if it already exists.
  • [104272] Staging and pre-running hooks abort if they return a non-zero exit code (as in v2.10).
  • [105590] Longer default health check timeouts. Modified settings will need to be reset after an upgrade from v3.4.1.
    • cloud_controller_ng/maximum_health_check_timeout is now 1800 seconds (30min)
    • dea_ng/default_health_check_timeout is now 300 seconds (5min)
  • [104166] app_store and app_mdns cannot connect to redis after restart
  • [105702] AOK: LDAP fix (omniauth-ldap)
  • [105430] Update to latest available kernel
  • [105432] AOK: AD LDAP auth failing - Encoding::UndefinedConversionError, "x82" from ASCII-8BIT to UTF-8
  • [105410] Better Docker base image update/upgrade instructions
  • [105365] Make all periodic-timer intervals configurable and responsive
  • [105372] Stackato-tty crashes if VM window confines text rendering
  • [105339] Prevent use of routes already deployed in different Org or Space
  • [105335] Problem with file view when using a load balancer (407 error)
  • [105333] Cloud Controller memory leak
  • [105246] Cannot delete migrated (3.2 to 3.4) applications
  • [105148] Updated cf-services-connector-nodejs to match current service API
  • [105110] Fixed rack support in Legacy Buildpack
  • [105066] Run app/.profile.d scripts being run with bash instead of dash
  • [105049] Updated recommendation for minimum disk size (30GB)
  • [104972] CSS delayed visibility in customized web console
  • [104963] Org/Space Customization does not work when a non-default quota is specified
  • [104898] Setting up Postgres external service changes internal Postgres password
  • [104882] Exceptions are getting wrapped up in unhelpful "An unknown error occurred" exceptions
  • [104817] Hastebin app: use filesystem service for storage instead of redis
  • [104807] apptail configuration problems showing in Cloud Events of upgraded systems
  • [104796] Application view button on spaces Apps view does not work
  • [104794] PostgreSQL external service - error after kato restart postgresql
  • [104787] Cannot deploy to - "Cannot allocate memory - unzip -l"
  • [104782] stackato report support help in console has incorrect syntax
  • [104757] Upgrades - exit out of an upgrade as early as possible if all nodes are already on the latest version of Stackato
  • [104728] Two apps can communicate only under limited circumstances
  • [104702] uninitialized constant VCAP::CloudController::App::InvalidRouteRelation
  • [104694] Staging cannot find admin_buildpacks directories on sandbox
  • [104692] Legacy import sometimes creates routes with a leading '.'
  • [104689] Disk usage after upgrade increases by ~5GB
  • [104680] Changes in app_bits_controller.rb lost in 2014-05 merge
  • [104664] AWS upgrade from 3.2.1 fails at Buildpack phase
  • [104635] Hardcoded strings in license settings view
  • [104627] Endless redirect loop in console when enforcing https on the api endpoint
  • [104605] cannot delete apps whose services got disabled
  • [104592] Deleting one user from a space deletes every user from that space
  • [104570] Applications > Versions: If app has never been started, there is nothing on the versions page
  • [104561] Imported Java applications do not auto-configure for rabbitmq
  • [104421] Retry different route if default route is unavailable during legacy import
  • [104358] %age calculations in bar usage graphs (e.g. DEA, AZ, PZ memory graphs) incorrect
  • [104077] App store app 'calipso' does not deploy
  • [103994] Push version info into the timeline
  • [103390] Inconsistent prefixes in app logs
  • [104672] Appstore urls do not get rewritten during upgrade from 3.2 that was previously upgraded from 3.0
  • [104560] nats-pub and nats-sub missing
  • [104470] Quota Definitions -> Quota Plans (CF terminology change)
  • [104487] Default space for ldap users should be generated based on user's email/login creds
  • [104396] "git clone" hangs in proxy environment

Stackato CLI Client 3.2.0

  • [105710] Ensured that filename in buildpack upload ends in ".zip" to avoid the moronic target-side type check by file extension.
  • [104659] Implemented handling of branded log prefix for system/app distinction.
  • [105708] Removed the admin patch command.
  • [105652] Fixed code ordering issue which caused use of an outdated client object.
  • [105644] Added SSL/TLS cert validation, and opt-out. Option --skip-ssl-validation bool. Env. Variable STACKATO_SKIP_SSL_VALIDATION. Sticky setting when used with target command.
  • [105637] Do not ask the user for a missing current org when the command spaces is invoked with option --all, as this mode does not require an org.
  • [105648] Generate a proper error message when the command delete --all is invoked without a current space, instead of crashing.
  • [105631] Extended the commands org-users and space-users with support for option --json.
  • [105192] Extended help for parameters based on the memspec validation type (units used, syntax).
  • Added new option --all to the command spaces. When used the command will show all spaces across all orgs, instead of just the spaces in the current/chosen org.
  • Implemented the restage command for applications.
  • [105490] Capture json and other errors in a bad --token-file, and report them properly instead of as internal error.
  • [105019] Tweak the help text of push option --force-war-unpacking, for clarity.
  • [103548] Implemented migration of applications across spaces and orgs.
  • [105497] Keep the original filename of a buildpack on upload, even when going through temp files (like used for uploading by url, and to remove a superfluous top-level directory).
  • [105518] Modified low-level REST code to reset an output channel on redirection, as a redirection may come with its own, unwanted, response body. Seen with buildpack downloads from github, adding a redirection message in front of the retrieved zip archive. While not breaking the archive this is a bug waiting to happen for other places.
  • Fixed ordering issue in the cli specification which caused it to drop group information when talking to a 2.x target during app name validation, and wrongly failing that validation.
  • Global options and proper option argument types in help texts. Option --colormode.
  • Support for feature flags, security groups, and space quotas.
  • Command push (as update). Added generation of sensible output for when 0-downtime switch-over was available and got triggered.
  • Added undocumented debug option --keep-form to commands create-buildpack and update-buildpack. Taking a path argument its use allows introspection of the form-data generated and uploaded by the client.

v3.4.1 (Jul. 29, 2014)

New Features

  • App Versions: User application updates now tracked with versions for easier rollback.
  • Admin Buildpacks: Administrators can now easily add custom buildpacks to the system.
  • [103539] Stackon subsystem for Docker-based Stackato add-ons.
  • [99935] IP based ACL for the Router.

Enhancements & Updates

  • Upgraded Docker to 1.0.
  • New upgrade back end for kato node upgrade (Sentinel).
  • Several system processes (e.g. AppStore and Redis) now running in Docker containers.
  • [103658] Logyard communication faster - using Unix domain sockets instead of TCP loopback.
  • [103789] Upgraded gnatsd to 0.5.2.
  • [101897] zeromq 3.2.4 for Logyard or appstore.
  • [103674] Custom AOK endpoint URLs.
  • [103908] Upstream merge of Cloud Foundry sources.

Application Stack Changes

  • [103517] git 2.0.1
  • [103301,104488] Upgraded to ActivePython and
  • npm 1.4.20 for Node 0.8
  • [104164] Include prerequisites for 2.10.x import
  • [104401] Added libevent-dev and libffi-dev
  • Improved sshd logging
  • [103130] App containers should have a stackato client

Bug Fixes

Management Console

  • [102977] Cannot issue patches to web console
  • [103892] Patch notification box should not be colored red when no patches are available
  • [104176] The App Store install button does not disappear when no app store URLs exist
  • [104026] Status: 400 error while inputting symbols in the search bar
  • [103579] Deleting timeline comment throws an error
  • [103605] No route displayed for apps deployed from app store
  • [104012] Application > Summary > Memory Usage: Should be Bright red if over capacity
  • [103934] App view: memory usage is counted for all stored droplets, not just the current one
  • [104021] Search bar on cluster management is confusing
  • [102716] Top bar re-design
  • [102966] Add quota usage dashboard to org view
  • [102949] Consistent search and filtering controls
  • [102369] Show available patches and major updates
  • [102718] Filterable App Store
  • [103739] OEM white-label customization improvements
  • [102963] Consistent terminology and language
  • [104169] Web UI errors with LDAP admin group
  • [103768] Adding/deleting repo creates duplicate "Allowed Repos" div
  • [104274] User Provided Service causes error when viewing app in console: 110003 - service plan could not be found
  • [104466] Timeline: Deactivate the preview button when the content is cleared
  • [99801] Internationalization efforts
  • [96728] Browser layout bugs on small viewports
  • [103313] App store deployment fails - Response: undefined
  • [101682] App Store sorting is confusing
  • [104391] Add a "view app" icon to Applications page
  • [104292] Cannot change app in timeline
  • [104058] Deploy app space drop-down list shows blank
  • [104322] Load custom styles before displaying "Loading" view
  • [104275] Drilling down on user provided services in web console loads no data
  • [104262] Update German translations
  • [103685] Apps list shows apps as "STARTED" when they are not
  • [103736] Clicking alerts goes to a page where there appears to be no problem
  • [104116] When admin alerts clear, the user drop-down in the header stops working
  • [103779] Router stats counter grows out of bounds
  • [97540] App Store install progress window is not scrollable
  • [103486] Cannot deploy apps to the desired org/space if there are more than 50 orgs/spaces in the drop down list
  • [104457] Cannot input tag in timeline on organization page
  • [104397] App Store Deploy fields marked as required or optional
  • [104298] User Provided Service Instances not displayed in Services list
  • [104503] API endpoint cut off on cluster management screen
  • [102146] Add an Application menu option for a developer
  • [103599] Add bootstrap-accessibility JS lib
  • [102920] Let users know that timeline posts/comments are in markdown
  • [103602] Cannot create a timeline event for individual apps in the drop down list
  • [104531] Non-started apps should appear as OFFLINE rather than STAGING
  • [104544] Need appropriate status on app view when app has a crashed instance


  • [103642,103657,103680] Fixed issues with kato relocate commands
  • [103763] kato node setup micro --no-start silently fails
  • [103719] kato always returns 0 exit code due to passing output through tee
  • [103869] kato data import of app fails and halts import
  • [103824] kato node reset factory leaves VMs in a corrupt state
  • [103919] kato relocate containers failing - Device or resource busy
  • [102928] kato node rename hangs waiting for password
  • [102768] kato restart on core node causes issues on other nodes
  • [103905] kato op max_client_upload fails while restarting cc_nginx - undefined method `join' for "cc_nginx":String (NoMethodError)
  • [103310] kato node reset soft has been removed
  • [103945] kato process ready --block does not actually timeout as expected
  • [97785] Refactor usage and cmd.rb files for kato commands with more than three sub-commands
  • [103906] kato op set_timezone not updated to use docker
  • [103675] kato data import ignores manifest.yml
  • [104238] docker daemon log should be in kato report
  • [103876] kato log tail drain should use UDP instead of TCP
  • [103823] Intermittent failure of kato patch when fetching patch manifest from
  • [103969] kato patch status --json error
  • [103676] kato data import does not always detect worker apps without a URL
  • [104469] kato node rename crashing
  • [104158] Patch state lost in micro -> node/core change
  • [101788] Audit API for validation of input
  • [101416] Add --upgrade option to kato op import_from_yaml_files
  • [103788] kato status hangs and stacktraces when a node goes offline
  • [104273] kato node retire -n <node> hangs, apparently waiting for a sudo password
  • [104078] kato report broken after 3.2.1 -> 3.4.1 upgrade
  • [103821] kato inspect returns 'Docker configuration is not valid Cannot locate docker image :latest'
  • [104093] kato node upgrade --skip-confirmation does not skip confirmation
  • [103744] The Kato::Info restricted field is always set to true

Upgrades & Migrations

  • [103720] kato node upgrade remote upgrades not re-throwing exceptions after creating upgrade-failed flat file.
  • [103921] Upgrade to 3.2.1 fails during role restart
  • [104403] Legacy import must ignore the app-dir in stackato.yml
  • [104374] Import fails if the app name contains an underscore
  • [104400] Apps with an empty manifest.yml are not being imported
  • [104405] Higher timeout for app import
  • [104360] Legacy import of standalone apps crashes
  • [103687] Upgrade failure with multiple CC's and cc_jobs process
  • [104315] Cannot login to console after upgrade to 3.4.1
  • [103305] Upgrades fail to work through proxy
  • [103742] Check 'urls' in addition to 'url' to determine the default route
  • [104402] Legacy import ignores environment variables
  • [104216] Docs: Limitations of legacy import
  • [103501] Improve upgrade output for end users


  • [104002] Staging has missing and duplicated lines
  • [103948] Restarting apptail does not tail existing apps unless they are restarted
  • [103107] App and system log stream improvements
  • [100913] Logyard drains targeting harbor services do not reconnect
  • [97378] Improve stackato logs using websockets
  • [104175] Kato custom log drains result in "MISSING"
  • [103790,103958] Split long lines of kato log tail and app log streams
  • [103150] apptail message not clear when dropping log lines
  • [103511] Fixed error colors in log tail
  • [103839] apptail rate throttling improvement
  • [103625] ERROR decoding json from a message with key
  • [104173] Legacy importer fails to remap services correctly when multiple services of the same name exist
  • [103108] cloud events occurring out of order in the app log stream
  • [103143] Improve apptail rate throttling


  • [103749] AOK: Apps can map the route aok.<system domain>
  • [104192] Fixed open redirect in AOK
  • [104280] Secure key regeneration using kato op regenerate ...
  • [103522] Reflected XSS vulnerability
  • [104020] Locked down supervisord


  • [104244] More detailed docs on prevent_x_spoofing router2g configuration with load balancers
  • [100770] Router file handle leak; systail inotify instances leak
  • [103944] Websocket issues
  • [103918] dispatchWebSocketErr function and terminate socket to allow long polling fallback for the clients

Staging and Legacy Buildpack

  • [103760] Updated
  • [104369] Import of legacy apps with mangled manifest.yml fails
  • [103600] Grails apps use different home directory in 3.2 (legacy buildpack) than they did in 2.10
  • [103699] pre-running hooks for imported legacy apps running too early
  • [103922] Do not setup STACKATO_HARBOR_* variables during staging
  • [103333] Docs: Procfile support
  • [103618] Docs: Client app config option --health-timeout and stackato.yml/manifest.yml support
  • [101636] Ability to set a custom npm registry blocked by staging plugin
  • [102356] Some buildpacks may not work with proxies
  • [103858] Staging error: cannot get instances since staging failed (400)

AOK (Authentication)

  • [104299] LDAP error on invalid characters
  • [103886] Login info endpoint 500
  • [102008] Buildpack assets should be cached


  • [104247] Improved documentation on configuring https proxies
  • [104270] Fix vendor_version information of /info endpoint
  • [103549] Org managers unable to create spaces
  • [101009] Eclipse CF v2 plugin does not work with Stackato 3
  • [104371] Incorrect redirect URI error after enabling application SSO
  • [103665] Assigning parent domain to org breaks CC_NG
  • [104326] Increase upload limit from 0.5GB to 1.5GB
  • [104114] Update to the new CF exception style in CCNG
  • [103320] Issues installing oracledb service in 3.2.1
  • [104141] Cannot create first user in multiple CC cluster - An unknown error occurred (10001)
  • [103830] Configurable http_proxy for appstore
  • [102399] supervisord pid problem with sudo
  • [103693] Documentation on buildpack Procfiles
  • [103895] Merge Cloud Controller changes from upstream
  • [102679] Allow admins to modify/theme the Client name
  • [103689] Implement lazy docker registry for sent
  • [99952] Upgrade rollback
  • [103462] Fix application instance downscaling
  • [103942] Cannot ssh to failed containers
  • [103727] Instructions for updating docker image are incorrect
  • [104153] Document container network security access options in fence
  • [103529] If fence/dea fails, report it in the app log stream
  • [103723] Default min_instances field not set
  • [103937] Account for missing /dev/fd when using docker 0.11.1+
  • [103836] Fence should report CPU usage scaled per core
  • [103399] Make fence delegate to Docker for memory limit
  • [100953] No error given with malformed app store yaml file
  • [103679] Docs: Updated notes on url generation in the client.
  • [103712] App push: Error 10001: undefined method `include?' for nil:NilClass (500)
  • [103184] Autoscaling: Respond to large changes in average CPU more quickly
  • [103952] cloud_controller_uploads_access.log not rotating properly
  • [104344] Docs: Architecture diagram out of date
  • [103668] Autoscaling: Each app needs its own heartbeat counter
  • [103059] kato data import/export cannot clearly delete the previous first admin account
  • [102689] core node does not recognize elasticsearch custom plug-in
  • [104171] Document required apt repo to install additional PHP modules
  • [102487] Fix docs for app_https_proxy
  • [103528] load balancer not working post-upgrade
  • [103638] Docs: Links to any URL are broken
  • [102793] Unicode in app description causes 500 error
  • [101282] Switch back to using upstream cloudfoundry/cf-uaa-lib
  • [98824] Check for ~/.stackato-firstboot-error in the MOTD
  • [103391] Connect to local harbor-redis instance on non-loopback address.
  • [104060] SESSION_AFFINITY prevents logins to the web console
  • [103677] Failed to create an instance of the elasticsearch service
  • [103606] JAVA_OPTS not passed into CF Java buildpack
  • [99109] API for all apps by group with all instances and instance usage
  • [104199] Fix missing Harbor metadata in a cluster import, when the harbor node is not import-local
  • [104340] service_bindings_url is incorrect for UPSIs
  • [104311] Regenerate cc API docs
  • [101350] Invalid bearer token: #<CF::UAA::InvalidTokenFormat: Not enough or too many segments
  • [104424] stackato-tty only shows partial version number
  • [97938] mssql service is not created during IronFoundry setup script
  • [102029] Firstboot rename does not catch non-zero exits from kato node rename

Stackato CLI Client 3.1.1

  • [104254] stackato tunnel fails against 3.2 server
  • [102774] Client hangs after target on Windows
  • [103099] Some client commands extremely slow past a certain number of users
  • [104052] Update test suite info for external users
  • [103899] Client always explodes *.war files
  • [104203] The quota attribute 'trial_db_allowed' cannot be set anymore.
  • [104225] Use a URL to specify the buildpack in create-buildpack
  • [103560] Inconsistent quota handling affects app push
  • [103703] cannot find package cmdr::history
  • [104145] client is polling for logs
  • [103586] client asks the user to use map-domain in 3.2+
  • [104389] Trap and ignore broken redirections issued to POST /bits (push, buildpack)
  • [103547] Rephrased help text of "create-user" (aka "register")
  • [103737] Detect and elide HTML dumps in general (http) error (status 500, etc.)
  • [103588] update-service-broker should offer to change URL, username & password attributes
  • [104137] Unable to locate service plan matching <service name>
  • [104279] Client needs update-service for user-provided services
  • [103683] Admin status not respected in create-space
  • [104025] marketplace command
  • [103530] Client reports failed staging for timeout start
  • [103984] Crash with ambiguous options to stackato delete
  • [103786] trace outputs garbage characters
  • [104415] Failure to delete services
  • [103578] Client should display URLs with https://
  • [104120] stackato buildpacks display bug
  • [102411] CLI help output does not show binary name
  • [103866] `` curl`` is hard coded to always request /info
  • [103562] Remove --distribution-zone in the list of stackato help push
  • [104150] stackato user-info is broken for non admin user
  • [101377] Stackato client commands returning Error 306
  • [103555] --space option is strictly enforced with --no-prompt but not without
  • [103947] logs -f alias for logs --follow
  • [103597] Org manager cannot unlink or change org user roles with client
  • [103590] Support raw PUT data with stackato curl
  • [103853] stackato services fails against 3.0.1
  • [103924] Need a --droplets option for quota create and quota configure
  • [103684] stackato logs --tail does not stream the logs
  • [104370] Client defaults to zero for quota values not explicitly set, should default to a useful value
  • [103713] Incomprehensible space-base error message
  • [103774] stackato tunnel fails with error - got keep-zip, expected parameter name
  • [104016] stackato push --as ... does not work without stackato.yml
  • [103900] Client should use the new logs API (websocket)
  • [103662] Added CLI option --domain, in parallel to the 'domain' manifest key.
  • [103854] client help --json output broken in 3.0.8
  • [104098] Unhelpful error for delete-service as admin
  • [103587] Crash on v2 broker service --json / credentials field missing from services output
  • [103581] Implement 'service-plan-visibility {public, private}'
  • [103576] Issues with domains on orgs without spaces
  • [103507] Add health_check_timeout setting during stackato push
  • [103286] Add a history command
  • [103912] Add a --code-only option to stackato rollback
  • [102364] Specify .war file with push --path
  • [103751] Implement {create,delete}-shared-domain <domain>
  • [103845] open should deduce the app name from the directory name
  • [103554] Use the same *-service-broker nomenclature as the cf client
  • [104291] files gives "~" rather than "~/app"
  • [104323] Changes to service-plans and help service-plan output
  • [103700] Do not warn for hooks::legacy-running key in stackato.yml and manifest.yml
  • [104383] create-buildpack - support directory as source of the BP (auto-create zip for upload)
  • [102848] Add support for admin buildpacks to command line client

v3.2.1 (Mar. 31, 2014)

Enhancements & Upgrades

  • Several UX improvements in the Management Console
  • Improved application distribution
  • Upgraded Docker to 0.7.6
  • Upgraded Go to 1.2 (for logyard, appstore, appmdns)
  • Improved kato node upgrade to handle interruptions
  • [102771] New app events (for example, crash) added to the application log stream
  • [102406] kato node reset now resets services data
  • [102336] Added kato op regenerate ssh-keys
  • [102916] Specify a 'default' org and space for new logins
  • [102511] Merge with latest upstream dea_ng/cloud_controller_ng

Application Stack Changes

  • New Docker image (stack) naming convention: stackato/stack/alsek becomes stackato/stack-alsek
  • [102193] Added logrotate
  • [102755] Removed build-time apt sources
  • [102483] Purged popularity-contest package
  • Upgraded Go to 1.2

Bug Fixes

Management Console

  • [102352] Search box broken on Admin -> Services
  • [102482] Load balancer role should only be enabled via kato
  • [103454] Web UI displays dashboard improperly when default zones are deleted
  • [102714] Sortable list views
  • [103171] Dead click space in green buttons on welcome page
  • [102795] Web UI does not handle unauthorized LDAP group error
  • [102551] Broken links under timeline
  • [102624] User icon no longer a (broken) link for non-admin users
  • [102280] Individual Application page does not say what type of service the service is
  • [103212] Showing apps as started while they are still staging
  • [103205] Deleting apps from console with services throws errors
  • [102440] Timeline events leaking between orgs
  • [102189] Non-manager users prompted to create space
  • [102575] Disabled services have check marks under app store
  • [103078] Help text for Space creation on first user setup screen
  • [101989] Add cd node-env to Clone Repo commands
  • [102518] Hide the navigation bar on smaller screens when user is an admin
  • [102095] Entering an invalid value twice on app settings makes input box disappear
  • [102291] Set window title to default product name/company on first setup
  • [102625] Remove the delete button from the currently logged in user


  • [102661] kato node reset failing to remove Docker containers/images
  • [102826] kato patch now respects proxy settings
  • [102983] kato op remap_hosts fails after import
  • [102406] kato node reset does not delete services
  • [102661] kato node reset soft ends with bash syntax error
  • [102568] After configuring LDAP, kato node reset does not change the setting to default
  • [102502] kato role add rabbit fails to associate with CC (403)
  • [102494] kato node remove hangs for 2-3 minutes if node is unreachable
  • [102584] kato config per-node
  • [102859] kato patch update now updates on all nodes by default(--local and --node to override)
  • [102226] kato status crashes when an external node initially becomes unavailable
  • [102353] kato op upstream_proxy set should prompt to restart DEA, not Stager
  • [102140] Make kato node remove a batch operation
  • [102167] Better error handling for invalid kato sub-commands
  • [102297] kato attach crashes if a unreachable IP address is given
  • [102593] kato export prompts for password when exporting PostgreSQL service
  • [102591] No error message for adding already existing data-services
  • [102136] Handle node attach error when unable to connect to redis
  • [102676] Staggered kato patch cluster install


  • [102706] Adding files to log stream with STACKATO_LOG_FILES
  • [103060] Document STACKATO_LOG_FILES in the comprehensive env var list


  • [102844] Remove express.bodyParser and use Connect.json instead
  • [103173] XSS vulnerabilities with org names
  • [102483] Remove popularity-contest package from image


  • Raised default container process limit (max_container_processes) from 50 to 256
  • [102472] dir_server process FATAL on isolated VMs
  • [102660] kato relocate droplets/containers leads to apptail error during push - ERROR -- No valid log files detected for tailing
  • [100412] Cleanup leftover temporary volumes created for appstore containers
  • [100671] Pass configured http(s) proxy to app store deployments
  • [102672] Memory leak in cloud controller
  • [102659] Failure to start DEA node in a cluster
  • [102670] Breakage in unsupported-runtime-detection patch
  • [102914] Duplicate user creation during import when using LDAP
  • [102542] v3 reduced fault tolerance in router
  • [102525] Cannot map URLs with a subdomain of less than three characters
  • [102243] Built-in Ruby buildpack continues to use Ruby 1.9.3p327
  • [102521] stackato_rest added drain uses loopback IP in cluster setup
  • [102224] Controller crash during repeated client pushes
  • [102572] Random segmentation faults when pushing a Ruby app with a legacy buildpack
  • [102770] crashed apps are not reported by stackato crashes
  • [102269] maintenance_mode fixed
  • [102334] Java-buildpack does not work behind proxy server
  • [102552] Error with LDAP strategy and no email address in LDAP
  • [102365] Legacy Buildpack: PHP apps are not getting a bound url
  • [102340] Incorrect DSN URI / database name field for mongodb
  • [102472] dir_server does not work on fully isolated VM
  • [102994] Mapping App URLs section describes 2.10 behavior
  • [102971] stackato scp docs missing some helpful content
  • [102902] Router appears to not drop downed routes
  • [102556] Add stackato user to docker group

Stackato CLI Client 3.0.6

  • [102244] Set quota when creating an org
  • [102537] Missing some service broker functions
  • [102760] Add support for application description to CLI (including stackato.yml)
  • [103160] Client generates bad manifest sometimes
  • [102496] stackato apps --all does not work
  • [101956] stackato scp should behave more like real scp
  • [102429] Better error messages for entities that do not exist
  • [102519] stackato scp appname internal error
  • [102752] stackato help admin too verbose at top level
  • [102412] Renamed client shows "stackato" prompt in shell mode
  • [103098] --token-file does not create empty token file
  • [102203] stackato open tries to open http:/// when the app has no URL
  • [102295] stackato open command help clarification
  • [102535] stackato delete logs debug data
  • [102298] Switch to cmdr v1
  • [102190] Stackato v2 commands should have deprecation/"v2" notice
  • [102596] Client error message should be clearer
  • [102459] stackato link-user-org should not have --developer
  • [102239] stackato info must show that (v2) is the API version
  • [102529] User spaces being displayed properly
  • [102319] Fixed handling of -group for Stackato 2.x targets
  • [102585] Default alias delete-service-broker -> remove-service-broker
  • [102954] Cannot access target: Error: cannot read "mymap(r1558)": no such element in array
  • [103054] Better scaling of user validation to large number of users
  • [102933] YML file generated by stackato client does not create services with proper syntax
  • [102266] Org/space exists error message is not very friendly

v3.0.1 (Dec. 18, 2013)

Bug Fixes

Management Console

  • Updated Web Console for enhanced user and organization management
  • [100376] Support line breaks in env variables in Web Console
  • [99914] App Store: Apps with missing requirements (e.g. services) have disabled deploy buttons
  • [101029] UI for creating routes and associating them with applications
  • [101031, 101738, 101739, 102081] UI for managing domains
  • [101786] Added route management UI to application view
  • [101840] Update web console browser requirements
  • [101910] App Store offers options for which Domain to push to
  • [101942] Fixed organization quota usage bar
  • [102013] Prevent deletion of reserved URIs
  • [102023] Removed "Allow non-local URLs" setting. Domains are now attached to organizations.

Docker (Containerization)

  • [101853] Upgrade to Docker 0.7
  • [101893] Fix a Docker container memory leak
  • [101772] New naming scheme for app images (stacks)
  • [101813] Added networking tools (net-tools package) in app image


  • [101635] Fix inotify panic in systail
  • [101616] Updated inotify
  • [100670] Avoid camel-case in stream JSON keys
  • [101558] Limit the number of custom app logs
  • [101863,101887] max_user_drains quota replaced by max_drains_per_app


  • [101648] kato report and kato cluster --all (-a) options changed to --cluster (-c)
  • [101354] Add kato cluster upgrade functionality
  • [102106,101700] Fixes to kato node reset
  • [101798] Fixed kato process stop config_redis (and sup stop config_redis) hang
  • [101081] Config changes force a process restart
  • [101188] kato report no longer asks for sudo password


  • [98724] Added global admin hooks
  • [99950] In-place node and cluster upgrade
  • [98574] Added SNI support to Stackato router (SSL configuration)
  • [101993] App Store: use shell-escaped arguments when calling the Stackato client
  • [101560] Enable rotation of some log files
  • [101692] Remove appstore containers after use
  • [101993] Fix shell escaping of arguments in app store deployments
  • [101872] Deleting an app will now delete its drains
  • [100396,101370] Added import/export support from both CF v1 and CF v2-based systems
  • [101823] Upgrade Ruby and Rails components against reported vulnerabilities
  • [101973] Upgrade ActiveRecord to 3.2.16
  • [102025] Revoke AOK token on logout
  • [101822] Address NGINX CVE 2013-4547
  • [101925] Remove passwords from cloud_controller_ng logs
  • [102003] Enhancements for web console re-branding
  • [101917] Account for different exit status returned by Fence
  • [97541,101449] Made libpq-dev and bundler available in the container
  • [99476] Improve security of router SSL handling
  • [100687] Add max length validation to org and space names (64 chars)
  • [100900] Deny non-SSL requests to AOK
  • [101254] Update java-buildpack from v1.0 to v1.5
  • [101590] --env settings are always written
  • [101750] Add --url to appstore push API
  • [101980] The DATABASE_URL and POSTGRESQL_URL environment variables now use postgres:// instead of postgresql://
  • [101980] The JDBC_DATABASE_URL environment variable is now available (jdbc\:postgresql\://)

Stackato Client 3.0.0

  • [102085] Fix issues communicating with Stackato v2
  • [101761] Improve handling of UPSI vs. managed services
  • [101763] Add configurable --timeout for start and push commands
  • [101473] Warn about a BUILDPACK_URL without proper framework type
  • [100929] Dropped --name alias for --as
  • [90686] Add stackato run --all option
  • [101381] Improve semantic checking of YAML keys
  • [101702] Allow the use of plain domains (without host) for (un)map
  • [94022] Add pre-push hooks
  • [97575] Improve -d debug port handling
  • [101507] Improve handling of multi-instance app startup
  • [101443] Improve handling of switch commands against known orgs/spaces
  • [101812] Improve handling of non-JSON responses
  • [101859] Show events for spaces as well as applications
  • [101949] Default to URL based on current space
  • [100346] Fixed hang in stackato tunnel with CF v1 targets

v3.0.0 Beta (Nov. 6, 2013)


Management Console

  • Update look and feel, improve inline loading and responsiveness using websockets
  • [99505] New Activity Stream API and timeline
  • [94745,94274] New disk activity and disk space graphs
  • [98737] New dashboard for router metrics
  • [99452,100047] Updated App Store to use new API and YAML format


  • [100352,101383] Use Docker for containerization
  • [96266] Improve app startup flapping prevention under high-load
  • [99614] New default limit of 50 processes per container. Configurable with kato config under fence max_container_processes
  • [98668] Memory consumption during staging is now limited to 1.5 times the application's allocated memory by default. Configurable with kato config under stager app_memory_multiplier


  • Add kato node retire to gracefully shut down a DEA, moving its application instances on other available nodes first.
  • kato admin ... commands removed; run the corresponding stackato admin ... commands or the web console
  • [93195,101079,101195] New kato node upgrade command to support node-level upgrades
  • [98649] kato process stop fence now correctly stops fence process
  • [98390] kato node attach now checks for version compatibility when adding a node to a cluster
  • [99521] --no-stop option removed from kato role remove command.
  • [99745] kato config no longer supports node-specific config; --node option removed.
  • [100861] Extend kato node setup firstuser to require default organization


  • [99429] Add Ruby 2.0 runtime and remove Ruby 1.8.7 (EOL)
  • [99368] Fixed problem building Nokogiri 1.6 gem. Stager now uses using system libraries (NOKOGIRI_USE_SYSTEM_LIBRARIES=true).
  • [94620] ActivePerl 5.14 removed
  • Update to ActivePython and ActivePython
  • [99977] Update PHP to 5.4 for default runtime
  • [99840] Support legacy frameworks with built-in buildpack


  • [99435] New kato log drain status command for showing drain status
  • [98325] Move drains from doozer to redis
  • [98687] Add "AppGroup" field to app log stream
  • [98836] Add "HumanTime" field to systail log stream
  • [98870] Add app crashes or exits (for example, OOM killer) to cloud events
  • [98686] Add harbor service provision events to cloud events
  • [96827] Cloud Events patterns are now configurable (via kato config)
  • [99534] New read_limit setting for apptail (default 16MB) to cope with extremely large application log files
  • [99571] Application log drain URLs now properly sanitized (v2.10.6 'logdrain-sanitize' patch)
  • [100512] Fix a leak in growing TCP connections causing systail to crash (inotify panic), and logyard drains to malfunction.
  • [100507] New WARNING log message on drain retries
  • [98214] Improve handling of drain state transitions
  • [98998] Compile with Go 1.1
  • systail now works with logrotate managed files (for example, router2g-access.log)


  • [99874] Core services ported to CF v2 API
  • [97164] Micro cloud starts with Memcached, Redis, PostrgreSQL, RabbitMQ, and MongoDB roles disabled by default (enable via Management Console).
  • [98930] RabbitMQ updated to 2.8.7
  • [98457] RabbitMQ 3.1.3 available (disabled by default). Enable with kato role add rabbit3
  • [99518] Increased default filesystem service size to 500MB
  • [99444] Make mysql service compatible with Amazon RDS
  • [98902] MongoDB client updated to 2.4.1 in application container


  • [93889] Update core components to Cloud Foundry v2 API
  • [99842] Update AOK replacement of UAA to new CF v2 API compatibility
  • [101162] Update NATS message bus to use gnatsd
  • [97026,98612] Replaced Doozer with Redis
  • [100386,100732] Update Linux kernel to Raring backport
  • [100215] prevent crontab breakage caused by newlines in environment variables
  • [99604] New X-Frame header configuration option for router
  • [96349] Separate user and admin documentation

Stackato Client

  • [96623] Full support for CF v2 API, maintained CF v1 compatibility
  • Updated command set for CF v2 terminology changes
  • [96061] Enforce stricter interpretation of positional options and commands to disambiguate some commands
  • [98509] Support array of applications in manifest.yml
  • [100564] Support user provided service instances
  • [99376] Show status of user drains
  • [100254] Removed host subcommand

v2.10.6 (Jun. 10, 2013)



  • [98980] New kato patch command
  • [96962] New Google Apps authentication strategy for AOK
  • [96395] Upgrade ActivePython distribute version to the latest
  • [98481] Fix a postgresql readiness issue on 'kato data export'
  • [98785] Upgrade Nginx to v1.2.9
  • [98626] Fixed 2.6.7 to 2.10 cluster data import error
  • [98632] pyopenssl is now installable via PyPM without an ActivePython Business Edition license
  • [98602] Fixed orphaned Erlang beam processes on RabbitMQ service deletion
  • [98636] Fixed filesystem service capacity limits
  • [98626] Autoscaling: Fixed ArgumentError in get_datastore (1 for 2)
  • [98681] Correct ownership of /home/stackato/.ssh/known_hosts
  • [98715] The cloud_controller X-Accel-Mapping header is missing
  • [98722] Locked down redis_server port from container access
  • [98735] Management Console: Overview now shows number of nodes in cluster
  • [99133] App Store: re-enabled Services filter
  • [98875] Fixed cloud controller error when increasing app instances in Management Console
  • [98884] Cloud controller now sends router.unregister NATS message on shutdown or restart
  • [98928] Fixed incorrect status codes in router2g access log
  • [98942] Hooks with grails (java_web) framework fixed
  • [98590] Fixed kato status role reporting overlap
  • [98972] Fixed MySQL service metadata import
  • [98985] Less aggressive DEA autoscaling
  • [98997] Fixed potentially blocking operation during retry of TCP drains
  • [99006] kato op upstream proxy ... now correctly modifies the LXC template
  • [99025] Longer (and configurable) timeout for kato op regenerate postgresql
  • [99040] Management Console now unmaps URLs correctly
  • [98966,99012] fixed broken postgresql first boot task
  • [99145] Fixed an issue where services may not be deleted from the web console
  • [99153] Fixed Harbor service issues with UDP backend health checks
  • [99220] Fix for cloud-init Stackato 'rename' task failing
  • [99230] Fixed cloud-init failing to install custom apt-packages
  • [98997] Fixed potentially blocking operation (up to 10s) in drain:stop
  • Fence/DEA (app container management):
    • [98979,98990] More accurate disk usage monitoring
    • [99100] Fixed a DEA memory leak with droplet management
    • [98749] Fixed FATAL -- Cannot create DEA pid file: Process already running
    • [98700] Fixed traceback on DEA restart (parse_info!': undefined method `[]')
    • [98688] Fixed Filesystem::node: No such file or directory - /var/vcap/sys/run/fence.sock error
    • [97619] Improved the accuracy of (accounted) container memory usage
    • [99147] Ensure container creation failure does not crash fence
    • [99204] Fixed "No such file or directory" ENOENT error on DEA
    • [98934] Container cleanup fix
    • [99202] Fixed "cannot add new key into hash during iteration" warning on container cleanup

Stackato Client 1.7.4

  • [97359] New default behavior for stackato update preserves previously set environment variables. New --env-mode replace option enables the old behavior of removing or resetting environment variables.
  • [96962] New token command for interactive token-based authentication with external SSO systems.

v2.10.4 (Apr. 5, 2013)



  • [96008] Fix logyard crash and uncleaned drains sometimes triggered by kato log tail
  • [98244] Delete app drains on stackato delete
  • [97614] Improve apptail reliability
  • [98279, 98354] Configurable finite retry on drains
  • [98169] Configurable maximum limit on user drains
  • [98326] Support for named custom format for use in drain URIs
  • [97856] apptail: handle NATS disconnection
  • [98280] Fix retrying behavior to reset retry delays after some point
  • Simplified formatting of log records in logyard.log
  • [98354] Reduced the frequency of retry warnings (esp. for Cloud Events)
  • Stability improvements [98091, 96337, 98342, 98243, 98240, 98423]


  • Node.js
    • [97557] Updated node08 runtime (0.8.22)
    • [98068] Added node010 runtime (0.10.1)
  • Perl
    • upgrade to uWSGI 1.4.9
    • updated ActivePerl-5.14 runtime (
    • added ActivePerl-5.16 runtime (
    • latest App::cpanminus supports installation via URLs and from git
    • added support for cpanfile to specify dependencies
    • added experimental support for Carton, the Perl module dependency manager
  • Python
    • upgrade to uWSGI 1.4.9
  • Ruby
    • [98316] Update Ruby autoconfiguration library for MongoDB
    • [98223] Improve handling of stdsync usage
    • [98113] Updated Ruby buildpack


  • [97806] Implement NATS connection error handling and retry logic
  • [98292, 98252] Fix possible JSON truncation in large responses
  • [98403] Correctly drop lost controller nodes from routing table
  • Additional fixes [96790, 98062, 98029, 97882]


  • [95989] kato users has moved to kato data users
  • [97251] Added kato node list to list all nodes and which roles are running on them
  • [98137] kato tail skips logyard INFO records unless --raw is passed.
  • [98107] Correct kato node rename to handle existing pushed apps when using mdns
  • [97934] Prevent kato data export exception if memcached services was never used
  • Additional bug fixes [98067, 96792, 97597, 97769, 97808, 97810, 97753, 97392, 97761, 97752, 97809, 97561, 98009, 96795, 98384, 98272, 98481, 98402]


  • [97520] Oracle DB add-on with stackato dbshell support
    • ORACLE_URL and JDBC_ORACLE_URL env vars if Oracle service present
    • Spring auto-configuration of Oracle DB binding
  • Core runtime updates for stability and security [97667, 97978, 97857, 97551]
    • [98515] Upgraded PostgreSQL to 9.1.9 to address CVE-2013-1899 (major), CVE-2013-1900, and CVE-2013-1901
    • [97645] Improved cookie handling of token
    • [97649] Prevent REST API cross site request forgery
    • [97646] Prevent XSS vulnerability through App Store yaml file
    • [98256] Address CVE 2013-1857 in Rails
    • [98521] Reduce controller default memory limit to 50% (from 70%) before it is restarted
    • [98050] Updated doozerd
  • [98281] Fix an issue with stopping apps during DEA shutdown.
  • [96175] Fix a regression in stackato files logs/staging.log not returning staging.log
  • [97895] Fix YAJL load error in wait_for_fs.rb (when filesystem service is used)
  • [98282] Speed up ssh connections to system
  • Web console improvements [97736, 97702, 95765, 97701, 96121, 97777, 98144, 97699, 98140, 97996, 97882]
    • [98254] Display Logyard configuration settings
    • [98042] Fix memory leak on long-open pages polling cloud events
    • [98127] Instant refresh of in-page modified components
  • [97025, 98259] Improve DEA internal connectivity speed
  • [98281] Improve DEA app cleanup on shutdown
  • [93626] Enforce random postgres password at first boot
  • [96007] Fix JBoss database support error when no services are defined
  • [97731] Reduce long polling calls from the web console
  • [97772] Correct ability to do first user setup after kato factory reset
  • [97993] Made log rotation more strict (rotate any log in /s/logs/ over 10MB)
  • [91120, 98299, 97973] Usernames are now stored in a case insensitive manner
  • [98182] Updated Mono framework add-on to 3.0 Beta (Mono version 2.11)
  • [98312] Update user authentication check to return compatibility to vcap_java_client
  • [97322] cloud-init updates
  • [97253] Change of "cloud_controller" role to "controller", new "primary" role
  • [98314] Restrict kernel messages to tty console to errors only
  • [98289] Improved console tty startup process to show progressive start state
  • [98288] Update MongoDB to 2.4.1
  • [97322] Improved cloud-init support to allow some kato commands to be called deferred
  • [98523] Make MySQL service STORAGE_QUOTA_INTERVAL and KEEP_ALIVE_INTERVAL configurable, move quota check to every 30s default
  • [97760] Properly handle timeout errors when checking DB quotas
  • [97619] Updated app instance memory calculation to use container group stats
  • [98550] Set container memory limit exactly at app instance memory request (was overallocating 12.5%)
  • Documentation updates

Stackato Client 1.7.2

  • [98187] New stackato admin grant/revoke/list
  • [98145] Fixes to some false negative key warnings
  • [98221] Fixed multi-app push from stackato.yml
  • [98045] Correct limits management between users and groups
  • [96810] Correct handling exploded .war directories
  • Additional bug fixes [98238, 96930, 98181, 92751, 98089, 97958, 97906, 97520]

v2.8 (Feb. 26, 2013)

New Features

App Store

  • Now a separate component.
  • Full streaming of application deployment output to the Management Console
  • [96401] App Store output now available in app log stream (stackato logs)
  • [97622] App Store uses upstream proxy info (e.g. provided by kato op upstream_proxy set).

Harbor Port Service

  • Allows external ports to be forwarded directly to your app
  • Supports TCP, UDP and dual TCP/UDP ports.
  • Supports multiple app instances (TCP only)
  • [95358] Supports use of custom and multiple SSL certs with your application. For apps that use frameworks, harbor provides a built-in HTTPS server with SPDY support for the custom certificate(s).
  • Passive health checking of unavailable backends or unresponsive apps.


  • Fully non-blocking container management
  • Less memory usage, faster DEA/stager performance
  • Faster container startup
  • Addition of per-app enforced disk limits
  • Improved security via Unix user-level compartmentalization in the kernel (each container now has its own Unix user)
  • Support for any IP range for LXC containers (configurable)
  • Removing (deprecating) lxctrl processes (less memory use)


  • Optional external user authentication component
  • LDAP authentication support

Bug Fixes


  • kato start|stop|restart now manipulates roles, not processes.
  • kato enable|disable is now kato role add|remove
  • kato attach is now kato node attach
  • kato setup micro is now kato node setup micro, kato setup core is now kato node setup core and kato setup load_balancer is now kato node setup load_balancer
  • kato setup controller and kato setup node have been retired. Use kato node setup core and kato node attach instead.
  • Change to how kato drain add options are passed. Dropped --option key=value --option key=value in favor of a just key=value key=value
  • For kato role add|remove (previously kato enable|disable), --all-but and --only are now boolean flags that act on the list of roles given. Therefore kato enable --all-but dea,prealloc becomes kato role add --all-but dea prealloc
  • kato role remove no longer supports the flag --no-stop
  • kato rename is now kato node rename
  • Added kato role list to list all roles and number of nodes they are enabled on.
  • kato config now takes action keywords set, get, del, push and pop. This is not backwards compatible. Example is kato config set <component> <key-path> <value>.
  • kato tail is now kato log tail
  • kato drain is now kato log drain
  • kato process start|stop|restart can now be used to manage processes. However, managing roles via kato start|stop|restart is recommended.
  • kato ready, for checking the ready status of processes, is now kato process ready
  • Add --node to kato process ready to check status or process on any node.
  • kato config default output is YAML
  • kato debug lsdoozer has been removed. Use kato config get --flat instead.
  • The previous kato status functionality that showed just a list of processes and their statuses can now be found under kato process list
  • kato import is now kato data import.
  • kato export is now kato data export.


  • [97274] PostgreSQL: HSTORE (key/value storage) support in the PostgreSQL service.
  • Redis: upgrade to v2.6 (release notes)
  • Python: upgrade to uWSGI 1.4.5
  • Perl: upgrade to uWSGI 1.4.5
  • Java: JPDA debugging for java_web applications via Harbor port service.
  • Go: upgrade to the latest Go buildpack
  • Logyard updates
    • [96620]: Drains are now retry infinitely if disconnected.
    • [96338]: Added new fields to the JSON of application log stream: AppID (application ID), AppName (application Name), NodeID (host DEA IP of the instance)
    • Added a new file:/// drain for debugging purposes.
  • [96493,96498] UI improvements for non-admin users in Management Console.
  • [95986] New 'app-dir' key in stackato.yml to support deploying from a specific sub-directory.
  • [97013] No port lookup for app host in incoming HTTP header.
  • [95902] Management Console: Warn when deleting the last role on a node.
  • [96729] Admin-configurable web console login expiration
  • [96700,96653] Rails security fixes.
  • [96677] mDNS backend: avahi-daemon removed and avahi_announcer replaced with avahi_publisher and app_mdns.
  • [96225] Manual SSH key configuration on systems with separate Router, Cloud Controller, or Load Balancer nodes is no longer required during setup.
  • [96696] Allow simultaneous 'kato tail' sessions.
  • [96932] Fix certain /files routes e.g 'stackato files' from not being routed correctly.
  • [95576] Prevent CC from being blocked during a vSphere scaling operation
  • [96708] Major changes to kato commands and command naming:

Stackato Client 1.7.0

  • [97399] Added '-d' option to enable JPDA debugging via Harbor service.

v2.6.7 (Dec. 19, 2012)

Bug Fixes


  • [96341]: Fix --format not being respected in kato drain add
  • [96225] Manual SSH key configuration on systems with separate Router, Cloud Controller, or Load Balancer nodes is no longer required during setup.
  • [94390] Added Node.js v0.8.14 as a dual runtime along with v.0.6.20
  • [96231] Java 7 has been added as a new runtime.

Stackato CLI Client 1.6.2

  • [95509] References to 'stackato' in renamed client help fixed.
  • [96386] Fixed output problems with 'stats' command.
  • [96427] Fixed problem with '--no-tail' option.
  • [96429] Fixed "Ambiguous runtime" error with 'standalone' framework.
  • [96438] Added option '--token' for direct specification of the auth token.

v2.6 (Nov. 27, 2012)

Bug Fixes


  • Application log streams
  • Redesigned to run on large-scale clusters with no single point of failure or inter-node traffic.
  • Better log forwarding via drains (tcp, udp, redis).
  • Added Splunk support (based on drains).
  • Key deployment events made available in the application log stream.
  • Improved documentation.

Cloud Events

  • Cluster support - now includes events from all nodes in the cluster.
  • Forward events via Logyard drains.
  • Build reports on top of cloud events via third party services.


  • new next generation cluster migration tool
  • kato import/export uses the cluster-aware back end formerly accessed with kato migrate (deprecated)
  • Bug fixes: [96065, 96072, 96073]
  • New kato history command shows a history of administrative kato commands ran across the cluster.
  • Removed Command: kato ls in favor of kato status --all


  • [93012] Allow multiple staging nodes
  • [96026] Create contrib plugin frameworks directory
  • [95935] Fixed misleading Cannot connect to doozerd warning in various cluster situations.
  • [96215] Security fix for app store installs
  • [96114] New dedicated Nginx instance for handling Cloud Controller file uploads
  • [94258] Configurable data services storage location
  • [95703] Fixed headers being prematurely sent on error pages with router2g
  • [96191] Upgrade to Ruby 1.9.3-p327
  • [96255] Fix issues with kato node rename when /etc/hostname is empty
  • Additional documentation for new features, plus edits and minor corrections

Stackato CLI Client 1.6.1

  • Application log forwarding setup via stackato drains add (user-level)

v2.4 (Oct. 29, 2012)

Bug Fixes

Application Logging

  • Logs are persisted between application restarts
  • stackato logs runs much faster, returns logs from all instances, and returns logs from stopped/crashed instances
  • stackato logs --follow tails logs from all instances

System Logging (Beta)

  • Aggregate stackato system logs from all nodes in the cluster
  • Output plugins to stream system logs to Loggly, papertrail, and others


  • New Command (BETA): kato migrate - next generation cluster live migration tool
  • New Command: kato op static_ip - intuitive tool for static-ip configuration
  • New Command: kato relocate ... - move key shared data to a new location (e.g. persistent storage)


  • MongoDB upgraded from v1.8.1 to v2.0.7
  • Node.js upgraded from v0.6.18 to v0.6.20
  • Ruby upgraded from 1.9.3-p125 to 1.9.3-p194 (with security patches from p286)


  • Import and export Users and Groups via web console (.csv format)
  • Simplified web console theming, with improved documentation
  • Upgraded Apache TomEE Plus from v 1.0.0 to v1.5.0. In addition to the support of new TomEE features as listed on the TomEE site, this upgrade addresses additional bugs:
    • [95426] Added Hibernate
    • [95082] Fixed Grails apps
    • [95425] Fixed test failure (Java apps with mysql services)
    • Other application compatibility issues
  • [95291] Fixed incompatibility with Eclipse Cloud Foundry plug-in and Spring Tool Suite
  • [94769] Allow non-web background processes with the "standalone" framework
  • [94734] Support non-web background processes for all other frameworks
  • [95491] Show DEA host node IP in application Details page for admins
  • [94076] Auto-Complete for adding users to groups
  • Experimental 'load_balancer' feature available via kato setup (BETA - not for production use)
  • Router2g improvements (BETA):
    • SPDY support on all HTTPS connections
    • Improved logging of events
    • Support multiple cloud controllers
    • [95280] Fixed not setting content-length for Django applications
  • Tuned system logging behavior for some components (DEBUG->DEBUG2)
  • Ability to deselect App Store URLs without deleting them
  • The App Store can now be disabled (removed from the side menu)
  • Allow auto-starting of apps deployed from the App Store
  • Documentation for new features, plus edits and minor corrections

Stackato CLI Client 1.5

  • [95702] Allow uploading very large applications (upload limits still imposed by server)
  • [92160] New subcommand scp to move data to/from app instances
  • Support for enhanced logging features
  • Client installation no longer available via pypm

v2.2.3 (Sep. 28, 2012)

Bug Fixes


  • [95661] Revert to older version of supervisord to fix an issue with cluster setup

v2.2.2 (Sep. 20, 2012)

Bug Fixes


  • Integrated Ruby buildpack. When using framework: buildpack in stackato.yml for ruby apps, BUILDPACK_URL can be omitted.
  • Java: Correct copy_source_files for non java_* frameworks
  • Java: Improved access to logs
  • Java: Add logs for debugging EAR JEE applications


  • New Relic integration docs updated
  • Updated cluster docs to use the correct form of kato attach and switched from kato setup to kato attach in autobecome
  • Added pyramid-default sample to Python docs
  • Updated Ruby deployment information
  • Updated autoscaling section with EC2 URL
  • Correct use of the Authorization HTTP header in client APIs


  • [95287] Improved memory usage calculation in forked Apache process apps
  • [95290] Tuned default Apache settings for app instances
  • [95458] Prevent doozer "TOO_LATE" warning
  • [95265] Handling client errors in App Store deployments more gracefully
  • [95314] App Store checks if application name is already in use

Stackato CLI Client 1.4.5

  • Fix Windows Unicode pathname handling
  • [95485] Tail stderr.log instead of startup.log on push in slow deployments. Latter file is not used by all frameworks.

v2.2 (Aug. 15, 2012)

Bug Fixes



  • Clarify usage of dnsmasq
  • [93980] Document how to configure DNS to support access to an application URL
  • Added .NET deployment section
  • Improvements in:
    • client/core API docs
    • language deployment (java, VCAP_SERVICES, DATABASE_URL)
    • stackato.yml docs (variable substitution etc.)
    • cluster & auto-scaling docs. (kato setup core)
  • App-store docs rewrite. (JSON -> YAML based store, simplified)
  • Best-practices docs rewrite. (migrations, export, import, dbshell...)
  • Deprecated Kato setup controller/node. See kato command reference.


  • [94844] stackato run dbexport exports mysql and psql databases
  • [94657] Changed the -n alias for kato setup -hostname to -a
  • [94357] Filesystem service split into filesystem_gateway and filesystem_node to make multiple filesystem service nodes possible
  • [94905] Adjust the VCAP_SERVICES service key to use an unversioned service name (version remains in label key) to bring parity to client service creation
  • [94783] Experimental router2g with WebSockets support
  • [94744] Updated doozerd binary, fixed verbose doozerd.log logging affecting disk i/o
  • [94828] Properly url encode console links for user names with special characters
  • [94501] Extract droplets asynchronously on DEA
  • [92457] Enable kato export via web console
  • [94638] Expose Allow Overprovisioning flag for all services
  • [94823] Switch App Store format from json to yaml and add tags field
  • [95204] Use core stackato client for App Store installation and make asynchronous [94834]
  • [95130] Auto redirect to logout when old or bad credentials are passed to deep linked page
  • [94734] Fix handling of file system links when viewed in web console
  • [95090] Defer re-staging of stopped apps when application environment is modified to next start request.
  • [95049] Ensure log rotation of cloud controller rails log
  • [94933] Improve handling of postgres restart in kato import
  • [95151] Fix auto-incrementing of uniq keys in kato import
  • [95156] Enable apt-get blacklist of some core packages to ensure system stability in system apt-get update
  • [93847] Set max_droplet_size via kato op max_client_upload
  • [95211] Run pre-running hooks after env generation during deployment to ensure database migrations are finished before the user commands execute
  • [95082] Fix Grails incompatibility with TomEE due to conflicting validation jar
  • [95028] Add JBoss framework JDBC service handling

Stackato CLI Client 1.4.4

  • [95197] Improved logout command to allow selective target logout
  • Fixed grails [95112] and aspdotnet [95122] framework auto-detection
  • Enhanced variable resolution in stackato manifest to resolve variables in the key portion of the yaml
  • Added EAR archive detection for JavaEE framework
  • Merge command line options from push into the stackato manifest values
  • [94852] Compare http headers case insensitively

v2.0.7 (Jul 19, 2012)

Bug Fixes


  • [94813] Fix for kato import failure when migrating from v1.x to v2.x
  • [94737] Fix for potential sudo errors when sudo is enabled in the containers
  • [94716] Fix for special characters causing issues in user email address
  • [94740] Fix for NATS timeouts in the Stager
  • [94578] Fix for Stackato usage reporting accuracy
  • [94847] Fix for more precise memory usage calculation in deployed applications.
    • Improved container cleanup management
  • [94836] Optimizations for DEA app queuing after cold startups of Stackato.
    • Configurable simultaneous app starts per DEA
    • Configurable maximum prealloc queue length
  • [94812] Improved first setup workflow for statically configured IP environments.
  • Various documentation improvements and updates

Stackato CLI Client 1.4.2

  • Updated the client

v2.0 (Jul. 10, 2012)

Bug Fixes

Cluster Management

  • kato, a better replacement for stackato-admin
  • Central multi-node configuration through doozer
  • Better process management through supervisord
  • CLI to edit configuration on all nodes: kato config
  • Fix: make nats-server run only on the controller node
  • Fix: automatically restart avahi-announcer and redis-server if they crash

Cloud Foundry Sources

  • New Memcached as a service
  • Router v2 upgrade
  • Configurable restart behavior for crashing apps
  • New Java Play framework, also supported via Buildpacks

Database Service Versions Upgrades

  • Postgresql upgraded to v9.1 (-> v8.4)
  • MySQL upgraded to v5.5 (-> v5.1)
  • Redis upgraded to v2.4 (-> 2.2)


  • Python
    • Custom pypm/pip options via $PYPM_OPTS and $PIP_OPTS environment variables
  • Scala support via Buildpack
  • Java EE 6 support
    • New java_ee framework via Apache TomEE
  • Integrated support for Iron Foundry .NET runtime
  • Runtime upgrades
    • Node.js v0.6.18
    • PHP 5.3.10


  • Allow stager to be run on different node
  • Avahi / mDNS now compatible with a cluster setup
  • Improved kato tail; now includes non-vcap components such as nginx, postgres, redis.
  • Base OS upgraded to Precise Pangolin (12.04)
  • Stability bug fixes.

v1.2.6 (May 15, 2012)

Bug Fixes


  • Fix for an issue using the stackato-admin update-services-ip command
  • Fix for missing Komodo debugging bits
  • Fix for a groups issue when migrating from Stackato v1.0 -> v1.2
  • Minor documentation updates

v1.2 (Apr. 27, 2012)

Bug Fixes


  • Upgraded Runtimes:
    • Ruby 1.9 upgraded to 1.9.3-p125 (from 1.9.2-p290)
    • Node.js upgraded to v0.6.15 (from v0.6.10)
  • Add Go language support
  • Add PyPy support (external buildpack required)
  • Add JRuby support (external buildpack required)
  • Python:
    • Allow pip to use mirrors (--use-mirrors) when installing packages
    • Upgrade to uWSGI 1.1.2
    • uWSGI now supports running Python 3.2 applications
  • Perl
    • Upgrade to uWSGI 1.1.2
  • Node.js:
    • All packages installed from NPM are now automatically added to the $PATH, meaning sudo privileges for 'npm install -g' are no longer required.
  • Clojure:
    • Upgrade to Leiningen 1.7; switch to Heroku Buildpack


  • Heroku buildpack support, with built-in buildpacks for Clojure, Go, Java, Play! and Python
  • stackato push|update will now tail the staging.log file simultaneously (like Heroku).
  • Support for user groups
  • Support for per-user and per-group limits
  • Improved dbshell: stackato dbshell is deprecated in favor of stackato run dbshell. The later works on cluster environment, and does not require local database clients to be installed.
  • Added STACKATO_SERVICES as an alternative to VCAP_SERVICES; PORT as an alias for VCAP_APP_PORT.
  • Added new service environment variables containing the respective connection URIs: DATABASE_URL, MYSQL_URL, POSTGRESQL_URL, REDIS_URL, MONGODB_URL, RABBITMQ_URL
  • Allow SSHing to crashed app instances
  • Optional support for caching staging files for faster stackato update
  • Disable truncation of vcap logs on component start. This preserves the log records when a vcap process is restarted.
  • Added support for OpenStack auto-scaling
  • Use geolocation-aware debian mirrors in application containers for faster apt-get calls
  • Improved the firstboot configuration process, eliminate a timeout bug in some cases.
  • Improved, faster VM boot process (Moved to Upstart).
  • VM now uses NTP for time synchronization
  • Fix for supporting externally mapped URI's on applications
  • Fix for Avahi broadcasting the wrong interface address
  • Automated DBGP Debugging
    • Pushing an app with --stackato-debug <host>:<port> enables Django, Rails, and Sinatra get requests to be debugged in DBGP-supporting apps like Komodo IDE with a STACKATO_DEBUG=1 parameter, and enables PHP debugging with the XDEBUG_SESSION_START=1 and XDEBUG_SESSION_STOP=1 parameters.
  • Updates to Stackato client.

v1.0.6 (Mar. 08, 2012)

Bug Fixes


  • Fix an issue for Filesystem services not unprovisioning correctly
  • Fix an issue with LXC container memory management causing a potential system lockup

v1.0 (Feb. 28, 2012)

Bug Fixes

Management Console

  • Increased the administrators control of the overall system
  • Ability for users to control their own applications
  • Core system server performance graphs
  • Improved cloud events logging


  • Python:
    • Run pip with requirements.txt, as that is the most common filename used by Python projects; if only requirements.pip exists, run pip with that file. For PyPM, use stackato.yml or requirements-pypm.txt.
    • Fix: make "-e" (source installs) in pip requirements.txt work
  • Clojure:
    • Automatically run lein deps to download dependencies
  • Node.js:
    • Upgrade to v0.6.10
  • Ruby 1.8:
    • Upgrade to patchlevel p-357


  • New persistent file system service
  • Support for the Cloud Foundry (vmc) manifest.yml
  • Added "Generic framework" to deploy applications with a arbitrary start procedure
  • Support for dynamically pre-allocating application containers.
  • make app environment variables available to staging, run commands, ssh and cron.
  • stackato-admin report: wrap logs and other system information for future diagnosis of a problem.
  • The applications timezone is now inherited from the Stackato VM host
  • $app_http_proxy is now available to specify a application specific $http_proxy
    • The new stackato-admin upstream-proxy command helps setting this variable, check the docs or stackato-admin help for more information.
  • Moved vcap logs in /tmp/vcap-run/ to $HOME/stackato/logs/
    • These logs are now compressed and rotated daily
  • show the actual staging log, instead of Error 310: Staging failed, upon push / update failures
  • allow top-level dotfiles during staging
  • stackato-admin become <node> now allows renaming operations on the VM, allowing you to specify the FQDN and local hostname.
    • The shorthand all role has also been added to stackato-admin become to assign all the available roles.
  • Bug fix for sporadic high load averages on DEA nodes
  • NewRelic Server Monitoring bits are now preloaded onto the VM.
  • Implemented fallback console on the virtual TTY1 for unsupported resolutions.
  • The TTY application now has fundamental error reporting built in.
  • git, mercurial and subversion are now made available in application instances.
  • Staging support for manifest.yml
  • Support for a external MySQL Database (e.g existing cluster)

0.8.2 Beta (Jan. 17, 2012)

New Features

  • Web-based management console for administrative control of Stackato
  • Merge with Cloud Foundry upstream as of Dec 21, 2011

Bug Fixes


  • Python:
    • Fix PYTHONPATH to contain correct value in the "run" command
    • Use PyPI mirrors during pip staging
  • Java:
    • Move from Sun/Oracle Java packages to Open JDK/JRE packages (why?)
  • Node.js:
    • Upgraded to version 0.6.7
  • Clojure:
    • Leiningen upgraded to version 1.6.2


  • Added dynamic first boot configuration process to randomize the hostname and SSL certificates.
  • New console application running on tty1
  • Micro cloud HTTPS/SSL certificate now supports the TLD and all subdomains
  • udev networking rules have been disabled


  • Application Staging now happens in LXC; thus more secure.
  • Support for establishing ssh connections to to application instances (experimental)
  • Improved and reliable stackato run based on SSH.
  • Support for schedule tasks through cron (experimental)
  • OpenStack build support

0.6.0 Beta (Nov. 03, 2011)

Bug Fixes


  • Perl:
    • Remove Perl 5.12 and make Perl 5.14 the default
    • Store uWSGI command line in a $STACKATO_UWSGI environment variable.
    • Store the default Perl application startup command in a $PROCESSES_WEB environment variable (TBD for other frameworks)
  • Python:
    • uWSGI for Python 3 is now officially supported
    • Store uWSGI command line in a $STACKATO_UWSGI environment variable.
    • Store the default Python application startup command in a $PROCESSES_WEB environment variable (TBD for other frameworks)
    • Add virtualenv's bin/ directory to $PATH before starting the application


  • Stackato admin dashboard (start with stackato-admin start dashboard)
  • Faster app deployment
  • Set user's home directory to correct location within LXC containers

0.5.2 Beta (Oct. 20, 2011)

Bug Fixes


  • Upgrade ActivePython to
  • Upgrade Node.js to v0.4.12
  • Upgrade Ruby 1.8 to patch level 290


  • Additional LXC fixes for non starting apps
  • Fix for broken clojure deployments

0.5.1 Beta (Oct. 3, 2011)

Bug Fixes


  • PHP support
  • RabbitMQ support
  • Python:
    • Properly show pip errors (during push/update/log) in staging.log
    • Add the application directory to $PYTHONPATH
    • uWSGI: remove --catch-exceptions by default
    • uWSGI: allow applications to spawn threads
    • uWSGI: do not spawn an unnecessary additional server process
  • Java:
    • Add :port back to VCAP_APPLICATION for the cloudfoundry runtime to work


  • LXC bug fixes and improvements (Error 306, etc.)
  • Speed up PyPM/PPM/cpan/pip by caching file downloads from the internet
  • Support for running custom hook scripts (post staging and pre running)
  • Upgrade to uWSGI (LTS)
  • Server process monitoring (via monit)

PDP4 (Aug. 25, 2011)

Bug Fixes


  • Clojure support
  • Python
    • Do not silently ignore packages missing in pypm repository
    • Deprecate virtualenv in favor of PEP 370 (faster deployments)
    • Allow invocation of scripts installed by package dependencies
    • Can now install packages with ./include dir. (for example, greenlet)
    • Drop 'Generic Python' framework
  • Ruby
    • Disable gem caching as it does not work with secure mode


  • Upgrade to ActivePython
  • Upgrade to NodeJS 0.4.10


  • Instances are isolated using LXC containers on DEAs
  • General improvements to error reporting in staging.log
  • Server now returns the git version info to client
  • Documentation is hosted locally at http://docs.stackato.local
  • stackato run
    • Include app environment added via stackato env
    • Wrap commands with quotes when necessary
    • Provide a dbshell command to invoke database clients
    • Run command no longer blocks the EventMachine reactor
  • MySQL
    • Use UTF8 as the default database charset

PDP3 (Jul. 13, 2011)

  • Support for PostgreSQL service
  • Fix random 306 errors when push/update fails
  • stackato run:
    • Fix 306 errors after ~11 seconds
    • Suppress traceback for ENOENT
    • Kill the process after timeout
    • Make PPM area directory available
    • Cleanup $PATH; add all runtimes to $PATH
  • Python
    • Note pypm/pip failures in staging.log
  • Node
    • Auto install dependencies using npm install


  • Upgrade to ActivePython

PDP2 (Jun. 22, 2011)

Bug Fixes


  • Python

    • pip and pypm can both be used to install Python


    • Python 3.x support for 'Generic Python' framework

  • Perl

    • Perl applications must have a top-level app.psgi file
    • Perl 5.14 support under the perl514 runtime name
    • Perl modules can be installed with cpanm via Makefile.PL
    • new Perl logfiles: logs/staging.log and logs/ppm4.log
  • Ruby:

    • Fix an issue with linking 1.9 gems with ruby1.8


  • Support multiple uWSGI binaries (PSGI, Python 2.7, 3.2)
  • No more uWSGI zombies
  • Installer creates stackato service script in /etc/init.d
  • stackato-admin controls the avahi-announcer's lifecycle (no more need to start/stop it separately)
  • Stackato client is installed on VM (necessary for multi-VM setups)
  • Resource-efficient avahi-announcer
  • avahi-announcer announce the apps names, not the mapped URLs
  • stackato-admin can manage admin users


  • stackato run:

    • Put runtime's path in front of $PATH

    • Close STDIN descriptor to prevent hanging processes

    • Support --timeout option (default = 1 minute)

    • Ruby: set GEM_{HOME,PATH} environment variables so dependent

      binaries (for example, rake) can be run

  • Remove the vcap/installer/remote after an install

  • stackato-admin uses yaml parser instead of grep

  • Install script is now called and accepts parameters

  • Do not regenerate the mysql password on restarting ./install

  • Do not pull uWSGI directly from mercurial repo

  • Allow git branch argument for ./install

PDP1 (Jun. 15, 2011)